-- Logs begin at Sat 2022-09-03 03:10:30 UTC, end at Sat 2022-09-03 04:09:49 UTC. -- Sep 03 03:35:07.642271 dv-cinder-799663-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 03 03:35:07.649988 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 3 03:35:07 2022] *** Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: nodename: dv-cinder-799663-12-iscsi Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: machine: x86_64 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: clock source: unix Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: pcre jit disabled Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: detected number of CPU cores: 4 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: current working directory: / Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: detected binary path: /usr/bin/uwsgi-core Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: your processes number limit is 47680 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: your memory page size is 4096 bytes Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: detected max file descriptor number: 2048 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: lock engine: pthread robust mutexes Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: thunder lock: enabled Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 03 03:35:07.653811 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 03 03:35:07.692425 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: Python main interpreter initialized at 0x5570cb7de9c0 Sep 03 03:35:07.692425 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: python threads support enabled Sep 03 03:35:07.692425 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: your server socket listen backlog is limited to 100 connections Sep 03 03:35:07.692425 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: your mercy for graceful operations on workers is 90 seconds Sep 03 03:35:07.692815 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: mapped 403077 bytes (393 KB) for 2 cores Sep 03 03:35:07.692815 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: *** Operational MODE: preforking *** Sep 03 03:35:07.693129 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: *** uWSGI is running in multiple interpreter mode *** Sep 03 03:35:07.693129 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: spawned uWSGI master process (pid: 93056) Sep 03 03:35:07.693350 dv-cinder-799663-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 03 03:35:07.694182 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: spawned uWSGI worker 1 (pid: 93068, cores: 1) Sep 03 03:35:07.694182 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: spawned uWSGI worker 2 (pid: 93069, cores: 1) Sep 03 03:35:07.694334 dv-cinder-799663-12-iscsi devstack@keystone.service[93056]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 03 03:35:11.960501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 03:35:11.960501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: from cryptography.utils import int_from_bytes Sep 03 03:35:11.961329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 03:35:11.961329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: from cryptography.utils import int_from_bytes Sep 03 03:35:11.961778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 03:35:11.961778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: from cryptography.utils import int_from_bytes Sep 03 03:35:11.962266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 03:35:11.962266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: from cryptography.utils import int_from_bytes Sep 03 03:35:12.010416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 03:35:12.010606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 03:35:12.010745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] command line args: [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 03:35:12.010986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 03:35:12.011109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 03:35:12.011182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 03:35:12.011242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ================================================================================ {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 03:35:12.011316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] command line args: [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 03:35:12.011453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] admin_token = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.011607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 03:35:12.011747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] config_dir = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.011832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ================================================================================ {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 03:35:12.011920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.011993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] admin_token = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] config_source = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] debug = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] config_dir = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] config_source = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] debug = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.012927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.013935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_config_append = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_dir = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_config_append = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_file = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_dir = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_file = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.014909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.015953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.016959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.017942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] publish_errors = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] publish_errors = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.018925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] standard_threads = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.019946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] standard_threads = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] use_journal = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] use_json = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] use_journal = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] use_stderr = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] use_json = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.020959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] use_syslog = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.021027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] use_stderr = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.021207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.021289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] use_syslog = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.021432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 03:35:12.021512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.021686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.021771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.021921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.022966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.023856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.024790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.025974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.026982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.027978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.028965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.029890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.030939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.031962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.032900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.033857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.034926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.035869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.036922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.037026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.037396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.037540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.037788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.038064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.038308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.038622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.038890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.039962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.040852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.041944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.042953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.043960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.044192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.044442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.044680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.044915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.045151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.045386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.045618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.045813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.045905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.046971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.047964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.048964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.049891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.external = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.050995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.password = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.051927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] auth.token = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.052963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.external = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.053887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.password = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] auth.token = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.054891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.055879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.056979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.057939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.058929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.059990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.060941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.061875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.062854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.063937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.064953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.065898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.066855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.067979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.068980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.069975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.070860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.071981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.072993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.073935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074481 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.074976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.075944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.076990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.077917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.078863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.079981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.080991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.081937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.082956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.083978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] role.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] role.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.084993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.085983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] role.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] role.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.086956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.087836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.088997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.089951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.090932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.091909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.092987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093072 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.093926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.094983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.095909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.096870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.connection = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.097974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.098948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.connection = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.099989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.100863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 03:35:12.101605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.101839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.102081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.102313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.102557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.102935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.103205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.103440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.103682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.104942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 03:35:12.105125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93069) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 03:35:12.135520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.135787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.135988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.139516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.139748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.139974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.144436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.148362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.172281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.172495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.174102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.174344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.224698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.224975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.273287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.273490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.273754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.273990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.429678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.429967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.430778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.453505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.453749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.453878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.453968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.454968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.455102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.455196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.465902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.466228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.478617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.478749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.478927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.479951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93068) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.480032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.480248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93069) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 03:35:12.496775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.496906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.496974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 03:35:12.497123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 03:35:12.497285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.497442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.497572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 03:35:12.497691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 03:35:12.497776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.497903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.498708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.508809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.509039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.515758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.515960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.518635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.518760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.518948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.519926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.520941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.521122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.521215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.521408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.521533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.543821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.543945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.545278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.545404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.545560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.545705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.550179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.550307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.550425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.550517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.550738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.550880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.557079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.557203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.557364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.557486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.562939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.563064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.563234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.563364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.563519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.563663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.563775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.563867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.564992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.580168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.580294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.580444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.580569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.584267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.584409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.587545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.587671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.587860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.587984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.592006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.592132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.595086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.595209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.598269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.598414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.598576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.598701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.598856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.598978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.599866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.615641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.615764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.615947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.616073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.616172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.616275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.616472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.616632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.623373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.623601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.624916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.625167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.626445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.626684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.626847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.626956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.629338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.629622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.629748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.629907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.630854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.641665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.641824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.641919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.642861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.643120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.643264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.643378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.643469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.643680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.658590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.658927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.661492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.661792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.664402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.664777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.665836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.666204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.667339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.667610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.667732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.667979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.668121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.668378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.674306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.674680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.675780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.676141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.678637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.679779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.680005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.687413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.687703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.687848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.687990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.688081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.688283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.694762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93069) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.695070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.695194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93068) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 03:35:12.695363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.695519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.695603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.695666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.695871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696615 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.696918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.697753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93069) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.698519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93068) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 03:35:12.731829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5570cb7de9c0 pid: 93069 (default app) Sep 03 03:35:12.732053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5570cb7de9c0 pid: 93068 (default app) Sep 03 03:35:12.735378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7163d-07d5-491e-898b-57731680960b None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:12.735664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7163d-07d5-491e-898b-57731680960b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:12.735935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7163d-07d5-491e-898b-57731680960b None None] PATH_INFO: `/v3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:12.740096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Sat Sep 3 03:35:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:19.326386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:19.326897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:19.327007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:19.333983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:19.339946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:19.340423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:19.340564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:19.410388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: INFO dbcounter [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] Registered counter for database keystone Sep 03 03:35:19.430670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG oslo_db.sqlalchemy.engines [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93069) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 03:35:19.442074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writer thread running {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 03:35:19.633478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG passlib.handlers.bcrypt [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93069) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 03:35:19.633771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG passlib.handlers.bcrypt [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93069) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 03:35:19.692764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:19.694418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:19.830526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 03 03:35:20.375566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:20.448269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: INFO dbcounter [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] Registered counter for database keystone Sep 03 03:35:20.472929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG oslo_db.sqlalchemy.engines [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93068) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 03:35:20.481107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writer thread running {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 03:35:20.598132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b7f31-77b7-4eea-a706-dae50a42790c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:20.603705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:20.604337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:20.604621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:20.604895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:20.614378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:20.665349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 03:35:20.665682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b94f7-a5e8-4c97-85c0-57234c909af8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:20.675741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Sat Sep 3 03:35:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:35:20.682177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:20.697297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319d1d-1fc9-4a08-baad-9c0373b29562 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:20.702961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:20.704105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:20.704389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:20.704667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] PATH_INFO: `/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:20.706385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] RBAC: Authorizing `identity:get_project(project_id=1fa3555aedbc4a4eaa2e41bd6d2eeab3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:20.758534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 03:35:20.758935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f30e40-6a23-4d83-b8ef-9b1ac3f12ce3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:20.763631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Sat Sep 3 03:35:20 2022] GET /identity/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:22.618676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89a7d-4ccb-4928-99d4-0865bf4fdc64 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:22.619514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89a7d-4ccb-4928-99d4-0865bf4fdc64 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:22.619514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89a7d-4ccb-4928-99d4-0865bf4fdc64 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:22.626291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:22.632626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:22.633029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:22.633182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:22.655232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:22.656753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:22.666257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 03 03:35:22.926296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca89a7d-4ccb-4928-99d4-0865bf4fdc64 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:22.941470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca89a7d-4ccb-4928-99d4-0865bf4fdc64 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:22.946646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:22.947280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:22.947558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:22.947830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:22.956908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:22.958702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4218a-e69c-4e77-870b-2790dd067ef0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:22.968524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Sat Sep 3 03:35:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:35:22.976851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:22.987305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2befef-2bad-4ef5-9945-7a504ebb5b96 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:22.993326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:22.993963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:22.994250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:22.994521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:22.996028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:22.997898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a99a-6971-41a6-8e07-f5e1724ce67d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:22.999486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Sat Sep 3 03:35:22 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:24.943310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99c43-aa06-45fb-ae6f-859b5909d22e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:24.943917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99c43-aa06-45fb-ae6f-859b5909d22e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:24.943917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c99c43-aa06-45fb-ae6f-859b5909d22e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:24.945223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:24.950410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:24.950888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:24.950990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:24.970386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:24.971616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:24.996659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a43c-e75f-430e-affd-92bfb3a8080c None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:24.997032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a43c-e75f-430e-affd-92bfb3a8080c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:24.997374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a43c-e75f-430e-affd-92bfb3a8080c None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:24.998771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:25.003721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.003984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.004259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.045990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG passlib.handlers.bcrypt [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93068) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 03:35:25.046377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG passlib.handlers.bcrypt [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93068) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 03:35:25.053403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:25.094621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:25.096274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:25.197416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:25.315258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.341788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:25.421984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07cd57-7a14-4ffa-868d-a53ba53db41e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.454766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.455374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.455676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.455936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.458399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.467304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:25.476149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:25.477130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:25.477951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:25.483141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.559376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d74b-171a-45ff-bfcf-8a9c90bb6355 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.561665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:25.563417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:25.581639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df133a-fdd1-4e66-846f-6d0f095cc592 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.598773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.599663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.599944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.600249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.615333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.615966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.616246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.616520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.617196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:25.617501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:25.618947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:25.630428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:25.636180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.669655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:25.671239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:25.735455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c44f-38f2-4d27-b090-e1d2a986edee None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.767965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:35:25.768114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.768670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.768953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.769226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.770395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:25.771957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:25.773516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:25.773628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.776602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:25.778839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.793756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:25.871183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2672-cf60-430b-9344-1bf390a42c36 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.877954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a98a35-969d-4e85-b9fd-2313a055951e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:25.905934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.906823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.907112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.907385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.917823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:25.918471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:25.918754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:25.919063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:25.920342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:25.921845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:25.924496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:25.926347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:25.927848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:25.930633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:35:25.948057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:25.949311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:35:25.950225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:35:25.950225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:35:25.950225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:25.950225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:25.955056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:26.010489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:26.017702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:26.037714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5305916-326d-4657-978b-96997b5f1d05 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:26.060286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ac1ea-a7f8-48a0-9411-9027d8439474 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:26.072969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:26.073577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:26.073852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:26.074117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:26.074865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.domain.id=default, user.id=5009c224905e4768b33c681c0f95b0f8) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:26.075169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:26.076604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f36af6-d9c6-4f74-87d4-cc70b81192df None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:26.083088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/role_assignments?user.id=5009c224905e4768b33c681c0f95b0f8&scope.domain.id=default&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:26.093840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:26.094541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:26.094851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:26.095133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:26.095812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:26.096158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:26.097611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a5a4d-df7b-4fa1-974e-24230e543edf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:26.118002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:28.044569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0f176-c779-4ee2-8b6f-cd9b0eb78f08 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.045124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0f176-c779-4ee2-8b6f-cd9b0eb78f08 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.045124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0f176-c779-4ee2-8b6f-cd9b0eb78f08 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.046449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:28.051729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7796-eb2f-4d1a-9dea-a624bd23aa06 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.052132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7796-eb2f-4d1a-9dea-a624bd23aa06 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.052324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7796-eb2f-4d1a-9dea-a624bd23aa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.079844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-91fc7796-eb2f-4d1a-9dea-a624bd23aa06 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:28.081068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-91fc7796-eb2f-4d1a-9dea-a624bd23aa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:28.196156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:28.502928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb075-c3b4-4ea1-9249-c34bceccadea None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.503224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb075-c3b4-4ea1-9249-c34bceccadea None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.503491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb075-c3b4-4ea1-9249-c34bceccadea None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.504824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:28.517326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.517617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.517896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.537166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e4826-230c-48e9-a22a-b6f2e08d2015 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.537568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e4826-230c-48e9-a22a-b6f2e08d2015 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.537847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e4826-230c-48e9-a22a-b6f2e08d2015 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.539223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 14/25] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:28.544488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f88b-0b68-4529-8c69-00c0985586db None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.544772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f88b-0b68-4529-8c69-00c0985586db None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.545048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f88b-0b68-4529-8c69-00c0985586db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.554678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:28.555889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:28.584662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6100f88b-0b68-4529-8c69-00c0985586db None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:28.585882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6100f88b-0b68-4529-8c69-00c0985586db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:28.646580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:28.688653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 12/26] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:28.694682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 15/27] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:28.735851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:28.841757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b382-2678-427a-b3f3-78c5eef32e46 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.842052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b382-2678-427a-b3f3-78c5eef32e46 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.842230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b382-2678-427a-b3f3-78c5eef32e46 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.843604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 16/28] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:28.850452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.851016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.851485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.866746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fa023-d64e-4f62-8dfe-e14d39bf539b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:28.888039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:28.889244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:28.934714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:28.935843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:28.936341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:28.936802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:28.958531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 17/29] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:28.967853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:28.975632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:28.976478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:28.977235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 13/30] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:28.984373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:28.990679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.032236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.099290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.158605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b62e-902a-4c78-b520-73c32abe8266 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.211217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f7a33-d0c4-4d44-9737-84169926923b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.221692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.222575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.223078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.223532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.224402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:29.224939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.226792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.239473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 18/31] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:29.247214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.284152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.284800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.285080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.285349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.305872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.308152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:35:29.331106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:29.332205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:35:29.333221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:35:29.333221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:35:29.333221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:29.334638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 14/32] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:35:29.336119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.353354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.392164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd633-f793-44df-960b-03bdf05ca793 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.428181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.428804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.429085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.429357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.430776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.431126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.432644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.435319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 19/33] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:29 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:29.436447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.446492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c974-a32a-4d2a-ba10-c9ba3114c508 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.493085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.493704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.494023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.494294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.494992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.496389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.553047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc007d4-04c9-4775-bfaf-13a3060fd904 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.588462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.589315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.589778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.590210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.591116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.592666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.596611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.626182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 15/34] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Sep 3 03:35:29 2022] POST /identity/v3/roles => generated 219 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:29.627415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.769670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 20/35] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Sep 3 03:35:29 2022] POST /identity/v3/roles => generated 225 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:29.773673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.874216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d21d-4ca9-4c76-952c-8a7ba7bc042b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.945322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.946552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.946990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.947269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.947957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:29.948268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:29.948657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d14179-e6df-4777-801f-8573f7897d23 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:29.949894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41253883-49d1-4eae-8cae-2a0da9ca0b9f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:29.969224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 16/36] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:29.971692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d023a-9147-4ec7-af02-7509b7c3b289 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.971980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d023a-9147-4ec7-af02-7509b7c3b289 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.972256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d023a-9147-4ec7-af02-7509b7c3b289 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.973676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 17/37] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:29.975699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d023a-9147-4ec7-af02-7509b7c3b289 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:29.988428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:29.995172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:29.995172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:29.995642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:29.999051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:29.999051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.001416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143be6f-ea23-464a-911d-e2aa8c4a2b64 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.007769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 21/38] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:30.009330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.009620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.009894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.035347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:30.037119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:30.113619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:30.116708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d023a-9147-4ec7-af02-7509b7c3b289 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:30.130034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 22/39] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:30.130894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:30.158774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:30.159422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.159706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.159981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.160898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.162442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e9b72-2cf6-43ab-9684-b0108c2ca0fc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.192517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 18/40] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:35:29 2022] POST /identity/v3/projects => generated 301 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:30.192517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b4868-0bce-444f-8647-cb0b7668ae07 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.192517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b4868-0bce-444f-8647-cb0b7668ae07 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.192517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b4868-0bce-444f-8647-cb0b7668ae07 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.195946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 19/41] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:30.204744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.205035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.205329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.225911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:30.227650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:30.249531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2359b5b-6c45-4c9e-88a9-8065d8c8896b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:30.297738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:30.298664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.298993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.299271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] PATH_INFO: `/v3/domains/default/users/5009c224905e4768b33c681c0f95b0f8/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.302139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5009c224905e4768b33c681c0f95b0f8, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.303678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.308310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:30.331413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 20/42] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:30.332897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 23/43] 10.222.0.25 () {58 vars in 1361 bytes} [Sat Sep 3 03:35:30 2022] PUT /identity/v3/domains/default/users/5009c224905e4768b33c681c0f95b0f8/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:30.403533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:30.518712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f65cc-bcab-4b56-9174-3666f8158ea6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:30.554100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:30.554739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.555054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.555331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.556448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.557905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.559672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 21/44] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:30.565172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:30.619787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:30.627133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:30.785649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396253a-af6d-4448-b44e-3ceb71c7fa59 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:30.798649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:30.825455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:30.826089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.826367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.826638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.827333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.828793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c99a82-4450-4cb7-a291-dc6f1dcf7aa4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.843255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397890-b7c3-4d51-ae4f-29610a905e2c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:30.859638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 24/45] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:35:30 2022] POST /identity/v3/projects => generated 312 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:30.885924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:30.886563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:30.886869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:30.887144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:30.887829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:30.889240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364182b-864b-44b0-9e2f-bc15a98d619c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:30.905442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 22/46] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Sep 3 03:35:30 2022] POST /identity/v3/roles => generated 223 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:32.265603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d9d1f-7d6d-4330-86e2-79b8bd62a246 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.266034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d9d1f-7d6d-4330-86e2-79b8bd62a246 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.266194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d9d1f-7d6d-4330-86e2-79b8bd62a246 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.267574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 25/47] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:32.273043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242af0-65de-4883-b9c5-a29644cd2173 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.273479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242af0-65de-4883-b9c5-a29644cd2173 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.273599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2242af0-65de-4883-b9c5-a29644cd2173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.292958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c2242af0-65de-4883-b9c5-a29644cd2173 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:32.294155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c2242af0-65de-4883-b9c5-a29644cd2173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:32.363760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 23/48] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:32.632336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d9d1f-7d6d-4330-86e2-79b8bd62a246 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:32.739310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d9d1f-7d6d-4330-86e2-79b8bd62a246 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:32.742229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:32.752086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8d8a-687c-427d-a126-23f84afc3d45 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.752245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8d8a-687c-427d-a126-23f84afc3d45 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.752431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8d8a-687c-427d-a126-23f84afc3d45 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.753803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 24/49] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:32.759176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.759458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.759734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.775438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:32.776071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.776348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.776612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.778414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:32.779636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:32.786735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:32.791573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:32.792540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:32.795711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 26/50] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:32.798845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:32.828981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:32.854022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 25/51] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:32.897039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49bb1-05f3-43f6-b367-ce7ecb2d6675 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:32.929418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:32.930030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:32.930306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:32.930578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:32.931279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:32.931582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:32.932982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:32.940863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 27/52] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:32.946453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:33.047965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998fd60-1692-4ec6-8453-2fda10b95842 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:33.081636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:33.082265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:33.082551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:33.082828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:33.084265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:33.085731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:33.087344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:32 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:33.093067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:33.114269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:33.122347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:33.160769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:33.195881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd320f-10b3-4e41-9f99-ba81d481178a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:33.212371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026430e6-3092-4492-9b9f-1e9b8d9ef21a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:33.227934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:33.228570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:33.228846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:33.229112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:33.230214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:33.231763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:33.233679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 28/54] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:33.239508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:33.246453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:33.247134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:33.247415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:33.247687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:33.248824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:33.250284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:33.251869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:33.257421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:33.330749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:33.338117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b9d93-468f-4fb0-bb3f-17305d3cc383 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:33.355655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59827a-a631-4f5e-83c7-769f93504a69 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:33.369144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:33.373662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:33.374280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:33.374565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:33.374860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:33.375594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.domain.id=default, user.id=5009c224905e4768b33c681c0f95b0f8) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:33.375905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:33.377372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4eb8f-3bd4-4333-be9a-8b407cd3bb66 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:33.383484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 29/56] 10.222.0.25 () {56 vars in 1322 bytes} [Sat Sep 3 03:35:33 2022] GET /identity/v3/role_assignments?user.id=5009c224905e4768b33c681c0f95b0f8&scope.domain.id=default&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:33.388886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:33.389514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:33.389796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:33.390071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:33.390735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:33.392246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369095f-53e2-41af-ada7-3e5e0f081da2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:33.424001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 28/57] 10.222.0.25 () {60 vars in 1140 bytes} [Sat Sep 3 03:35:33 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:35.330970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3c8f-2d2c-48a0-b04b-ed5142cfdfce None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:35.331735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3c8f-2d2c-48a0-b04b-ed5142cfdfce None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:35.331872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3c8f-2d2c-48a0-b04b-ed5142cfdfce None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:35.334019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 30/58] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:35.342859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:35.343382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:35.343455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:35.365635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:35.366896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:35.437536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:35.703070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d3c8f-2d2c-48a0-b04b-ed5142cfdfce None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:35.814569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d3c8f-2d2c-48a0-b04b-ed5142cfdfce None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:35.850753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:35.851404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:35.851674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:35.851939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:35.853072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:35.854518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077d047-c3fb-4bbb-908a-8a29029c659a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:35.856111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 31/60] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:35.862604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:35.963321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:35.969739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81af15-622d-46f8-bed5-6b85b6d8738e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:36.005962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:36.006563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:36.006843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:36.007139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:36.007777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:36.009210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf71e-adfd-41cc-a24b-f19b62f3cef5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:36.039759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 30/61] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:35:35 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:38.872116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee49ff-23bb-44fc-8cf1-dbed2f8b4439 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:38.872629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee49ff-23bb-44fc-8cf1-dbed2f8b4439 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:38.872629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee49ff-23bb-44fc-8cf1-dbed2f8b4439 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:38.874068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:38.879455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:38.879853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:38.880014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:38.900712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:38.901905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:38.949109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d55d2-a2db-4a95-8fe9-f3f02293d3c3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:38.949733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d55d2-a2db-4a95-8fe9-f3f02293d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:38.950198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d55d2-a2db-4a95-8fe9-f3f02293d3c3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:38.951761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 33/63] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:38.956826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688674-36fc-4e8a-9332-64520ee7a77e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:38.957361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688674-36fc-4e8a-9332-64520ee7a77e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:38.957830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688674-36fc-4e8a-9332-64520ee7a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:38.976344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-68688674-36fc-4e8a-9332-64520ee7a77e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:38.977961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-68688674-36fc-4e8a-9332-64520ee7a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:38.999566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:39.005589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 31/64] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:39.052543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:39.311619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:39.420074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60a3fb-7c25-475c-8ac8-6702b8429450 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.420836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60a3fb-7c25-475c-8ac8-6702b8429450 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.421300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60a3fb-7c25-475c-8ac8-6702b8429450 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.423540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 35/66] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:39.427405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:39.429976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.430474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.430981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.453352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:39.454900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:39.457551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95169e8-c120-4c6a-9957-47e462af0985 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:39.492922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:39.493562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.494012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.494430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.508124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:39.514413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:39.515378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:39.516310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:39.518716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 32/67] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:39.523799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:39.554830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 36/68] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:39.554830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:39.672701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:39.692627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab4872-8731-45e4-9b33-ca010c7c73ce None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:39.693132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bb194-eda0-42a5-8081-0d179c500600 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:39.728499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:39.729137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.729416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.729686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.736883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:39.738011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.738910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.739374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.740300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:39.741020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:39.741283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:39.742417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744aaa1-29d4-4d52-a03e-15e502f6b994 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-190468f6-91cc-420a-b942-f27f3d0205f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:39.746188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:39.747069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:39.747807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 33/69] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:39.748913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0979-0fdb-4c23-aac6-f4870b196a18 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.749180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0979-0fdb-4c23-aac6-f4870b196a18 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.749447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0979-0fdb-4c23-aac6-f4870b196a18 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.751348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 34/70] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:39.752462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8de58-8a62-4b8c-97fb-662b9db87479 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.752746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8de58-8a62-4b8c-97fb-662b9db87479 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.752997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8de58-8a62-4b8c-97fb-662b9db87479 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.754359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 35/71] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:39.755340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8de58-8a62-4b8c-97fb-662b9db87479 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:39.755683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 37/72] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:39.757072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.757497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.757891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.764964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:39.779647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:39.780829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:39.856108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 38/73] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:39.857207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:39.879104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8de58-8a62-4b8c-97fb-662b9db87479 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:39.913441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:39.914055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.914320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.914591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.915281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:39.915586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:39.916982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538daea6-c56a-4680-a939-2dfed0368152 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:39.929037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 36/74] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:39.934559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:39.934863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:39.935145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:39.946958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:39.953031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:39.954539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:39.971515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec045a-cb2a-4d04-992c-6f1f93756dbd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.008339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.009011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.009291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.009563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.024002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.025396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.026663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.027126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 39/75] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:39 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.027894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.054851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 37/76] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:40.054851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.139414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7f12-0a0f-4fbc-945a-bc53bd2cf0eb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.185534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2508-62bf-4464-a8fc-26cf65e7ebf9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.185793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.186552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.186864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.187136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.190020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.199437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:40.204404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:40.205224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:40.205975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 40/77] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:40.206616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.228953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.229600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.229878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.230148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.234003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.234003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.234772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 38/78] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.239545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.319835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.344319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524fff4-72a8-482a-acc4-e55a89785382 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.359550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7dbf-ebb9-47e2-bcd7-eab7340c43fa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.377119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.377759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.378037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.378305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.391923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.392586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.392864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.393141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.403215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:40.408617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:40.409551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:40.410305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 39/79] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:40.410996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.441100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.448951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.450437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.510143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318dac4-5661-4db1-9b91-b57972a678d6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.532037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.544183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.544808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.545084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.545356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.546017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:40.546322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.549913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 41/80] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:40.550049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.550137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.560239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 40/81] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.560239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.642358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.648959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8592e94-b980-40c6-935b-21562bd01b5f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.652870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8587-c75b-48b6-add0-1451c0bfb191 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.682231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.682873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.683159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.683437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.685056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.685697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.686004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.686281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.687458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.688927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.690499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 41/82] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.691258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.745422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.753271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.754747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.792262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6be6-f8e1-4253-b64a-8e15f8e08a89 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.824225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.824825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.825106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.825375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.826024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:40.826323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.827730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.835604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 42/83] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.836289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.849504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.856072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 42/84] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:40.856886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:40.940670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed7b96-fd42-4c6c-91da-7e13d2cf6361 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.955805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:40.965616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd215f-870d-42cb-940d-54aa7945d067 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:40.979904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:40.980661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:40.980949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:40.981228 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:40.982395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:40.984208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:40.985912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 43/85] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:40.986656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.000288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.000928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.001208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.001470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.002632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.004124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.005705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 43/86] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:40 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:41.006519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.066442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.085510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09097a-c158-4d97-b77a-f3e7633d1c11 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.105151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e75ce-ede3-4e5e-bff2-405cc4919137 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.118391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.119017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.119303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.119579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.120234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.121688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.139261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.139911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.140213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.140474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.141621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.143111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.143660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 44/87] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:35:40 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:41.144694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 44/88] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:41.145456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.147678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.192435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.264018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97dfb-524e-4a56-a54d-01d825f71917 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.270987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73463fc-606d-44e2-96f9-1b39c91a3a50 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.296609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.305981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.306595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.306919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.307202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.307948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:41.308262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.309735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.311750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.312458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.313193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.313484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.314963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.315725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 45/89] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=87b7a008126642369351bd284f3e8354 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:41.316564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.319857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.321657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 45/90] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:41.322466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.459904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa54b7-f9e2-427a-9ecb-529c38341273 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.467260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59358b-27ad-408b-bbaa-565f47e9a40f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.476622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.511115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.511751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.512039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.512317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.519668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:41.520568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:41.521113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:41.522034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:41.604186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.605788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.607156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.616004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:41.617514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:41.772778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 46/91] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:41.774386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.786618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.811406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 46/92] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:41.812272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:41.947406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de55dfd-0cb5-4694-82bf-8ad68142c5cf None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:41.984821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:41.998723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5746494-d875-4297-b86b-491be54cd28a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:42.001791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:42.002655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:42.003149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:42.003614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:42.005248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=5009c224905e4768b33c681c0f95b0f8) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:42.010269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:42.012037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:42.022129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 47/93] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/role_assignments?user.id=5009c224905e4768b33c681c0f95b0f8&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:42.023379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:42.083617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:42.084321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:42.084606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:42.084877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] PATH_INFO: `/v3/projects/b1f8e0cd5804485198d1bc117e18f553` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:42.091801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] RBAC: Authorizing `identity:get_project(project_id=b1f8e0cd5804485198d1bc117e18f553)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:42.093324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:42.095163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 47/94] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:41 2022] GET /identity/v3/projects/b1f8e0cd5804485198d1bc117e18f553 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:42.101999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:42.158612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:42.224573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d257-36b8-42e5-a59c-28b5cf2d877c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:42.299353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:42.299908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:42.300190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:42.300462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:42.302788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:42.304332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:42.307851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 48/95] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:42 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:42.311716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:42.327189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b05357-e871-42c5-8671-ed6011912bee None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:42.392671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:42.393295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:42.393569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:42.393835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:42.394628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=b1f8e0cd5804485198d1bc117e18f553, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:42.394961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:42.396463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005841f-12a6-4c8e-a662-1b2cf941d540 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:42.405924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 48/96] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:42 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=b1f8e0cd5804485198d1bc117e18f553&role.id=87b7a008126642369351bd284f3e8354 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:42.461733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489b7bc-43e0-4893-8cae-05383f4610b0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:42.498298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:42.498959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:42.499240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:42.499513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:42.500248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:42.500554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:42.502015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996b40d-e7db-47e5-9d91-45d49d92828d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:42.507904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 49/97] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:42 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:43.656917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46ed-75d7-4671-9101-cd722365abdb None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:43.657411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46ed-75d7-4671-9101-cd722365abdb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:43.657411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46ed-75d7-4671-9101-cd722365abdb None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:43.658735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 49/98] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:43.664130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c640-1459-4a70-9e1f-d11228d112e6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:43.664516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c640-1459-4a70-9e1f-d11228d112e6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:43.664674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c640-1459-4a70-9e1f-d11228d112e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:43.673617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:43.691786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4917c640-1459-4a70-9e1f-d11228d112e6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:43.693042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4917c640-1459-4a70-9e1f-d11228d112e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:43.737623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b579e-95ce-402e-b0e2-dd83760fa02c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:43.737920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b579e-95ce-402e-b0e2-dd83760fa02c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:43.738183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b579e-95ce-402e-b0e2-dd83760fa02c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:43.739775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 50/99] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:43.747283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da380d9-09d6-4cd9-a612-737f544f5bad None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:43.747563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da380d9-09d6-4cd9-a612-737f544f5bad None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:43.747835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da380d9-09d6-4cd9-a612-737f544f5bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:43.776599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 50/100] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:43.787474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7da380d9-09d6-4cd9-a612-737f544f5bad None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:43.788694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7da380d9-09d6-4cd9-a612-737f544f5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:43.839316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:43.889431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 51/101] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:44.154242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6515e-21c7-4cff-8ea0-959553c439fe None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.154608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6515e-21c7-4cff-8ea0-959553c439fe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.154791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6515e-21c7-4cff-8ea0-959553c439fe None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.156237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 51/102] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:44.162431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.162719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.163046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.188117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:44.189303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:44.203647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6515e-21c7-4cff-8ea0-959553c439fe None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.261045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 52/103] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:44.350587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6515e-21c7-4cff-8ea0-959553c439fe None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:44.387256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:44.395459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:44.396147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.396835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.397384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.412038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:44.416763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:44.417614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:44.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 52/104] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:35:44.423771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.446719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.579809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:44.592971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f519-e68c-4334-a77c-a7cb1c7930fb None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:44.647249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0a82-c10f-4f9e-8bc8-baa6a006e551 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:44.655570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:44.656211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.656502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.656804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.657494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:44.658033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:44.659287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:44.682652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:44.683165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 53/105] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:44.683263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.687079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.687425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.687705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.689372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:44.690869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:44.692456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 53/106] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:44.693260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.789670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:44.870957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841e79-2f76-416a-a713-f2370e61501f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:44.920913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b6301-4bf7-4613-827d-17a9032b70eb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:44.946622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:44.947427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:44.947707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:44.947980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] PATH_INFO: `/v3/users/26b045e8d8344e6e94db0f2036a418bd` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:44.977455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] RBAC: Authorizing `identity:get_user(user_id=26b045e8d8344e6e94db0f2036a418bd)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:44.978943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:44.987990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 54/107] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:44 2022] GET /identity/v3/users/26b045e8d8344e6e94db0f2036a418bd => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:44.988843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:44.999378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.000091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.000371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.000642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.025423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:45.030722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:45.031726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:45.032465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:45.038095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 54/108] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:45.038975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.062105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.145214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223cd2f-ff5e-4519-8ada-b5fb4ce2ad71 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:45.168565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.169463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8298ee-e569-4e25-8306-08f48cacd93f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:45.187218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.187839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.188128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.188404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.189065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:45.190542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:45.208427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.209019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.209310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.209591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.226852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 55/109] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:35:44 2022] POST /identity/v3/projects => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:45.226852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.329443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:45.331385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d3a29-57b4-4500-a578-1163e7a41869 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:45.345183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.357997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170acaa-34e3-4f70-b66a-1791b369fcd8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:45.404532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.405162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.405447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.405723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.406400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:45.406705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:45.408956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285c5dc-723b-4b71-875b-88a8bc83fc8a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:45.418153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 56/110] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:45.420059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6cfac-4302-4b87-ab17-bed36299d37f None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.420059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6cfac-4302-4b87-ab17-bed36299d37f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.420059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6cfac-4302-4b87-ab17-bed36299d37f None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.422686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 57/111] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:45.422686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7a5d9-1751-476c-8dbb-c3425a0d0199 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.422686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7a5d9-1751-476c-8dbb-c3425a0d0199 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.422955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7a5d9-1751-476c-8dbb-c3425a0d0199 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.424703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 58/112] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:45.425665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a5be-dedf-4294-93fb-a7799a161c16 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.425958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a5be-dedf-4294-93fb-a7799a161c16 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.426225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a5be-dedf-4294-93fb-a7799a161c16 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.427799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 59/113] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:45.428528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5a5be-dedf-4294-93fb-a7799a161c16 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.452464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 55/114] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:45.453552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e464-9bce-4c78-b5d4-524d863996ef None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.453838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e464-9bce-4c78-b5d4-524d863996ef None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.454112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e464-9bce-4c78-b5d4-524d863996ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.463684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.482208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9435e464-9bce-4c78-b5d4-524d863996ef None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:45.484083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9435e464-9bce-4c78-b5d4-524d863996ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:45.548424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5a5be-dedf-4294-93fb-a7799a161c16 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:45.558907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 56/115] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:45.560573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.560820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.561054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.584371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.585060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.585347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.585623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.586961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:45.587899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:45.588614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7db47-7569-4f69-8646-dfaaa57d1610 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:45.589537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:45.595418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 60/116] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:45.596828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.597167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.597450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.617233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:45.618311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:45.636186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.684732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 57/117] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:45.685875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.714874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 61/118] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:45.716775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.750872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:45.854849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaaf82-f861-4da4-a2b2-d0a14c4133f2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:45.900674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:45.901787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:45.904328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:45.904780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:45.906295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:45.908871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:45.911709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 62/119] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:45.913882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:45.957014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d801d4d-574f-429b-acd2-759a50154d7a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.021531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.024092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.024791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.025144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.025418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.027920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037a59-a33b-4079-bc5f-48fc7e874970 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.031580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.033614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.035546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 58/120] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.036491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.068237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.069275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.069741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.070246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.081707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:46.082674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:46.090247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:46.091128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:46.091902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:46.094209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 63/121] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:46.095197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.143842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad5a69-88fc-47ad-a186-add1a3d4939f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.179456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.180502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.180794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.181220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.183293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, user_id=1ccc32293d63478eb9e68ea2e8c599ba, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.184791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.199659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 59/122] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:46 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:46.200581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.205569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b1e0-4853-429e-b4ba-415c44632f73 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.249633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.250496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.256507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.256998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.257971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=26b045e8d8344e6e94db0f2036a418bd) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:46.258434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.260190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.266670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 64/123] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/role_assignments?user.id=26b045e8d8344e6e94db0f2036a418bd&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.267809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.295704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.339983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.343036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1d447-ecd0-4f9a-8ae2-748545dae9ab None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.433817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.434466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.434751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.435059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.435749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:46.436057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.437671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.438344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113be-d66b-49f3-be55-b0b5c68dfa91 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.450405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 60/124] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.451220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.486662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.487635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.488196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.488710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.500971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:46.507715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:46.508582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:46.509394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:46.511644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 65/125] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:46.512623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.587495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3f0c5-5fce-4f3a-8c22-77cf32da37d1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.596295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.616158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb18865-a566-433f-a562-40a7fbdd17ae None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.626072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.627134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.627589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.628127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.639792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:46.645942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:46.646826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:46.647711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:46.649996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 61/126] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:46.651287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.652868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.653590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.653942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.654289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.655544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.657098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.658996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 66/127] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.659738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.689477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.763432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4add3e0-8c18-4a41-959a-47510146ef74 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.801513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.802321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.802675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.803073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.803811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:46.806828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.806828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.816193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 67/128] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.817115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.847754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:46.854231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ca98-56bf-424e-931a-f39e0c818b7e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:46.916180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:46.916806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:46.917090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:46.917372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:46.918048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:46.918359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:46.919803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:46.931674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 62/129] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:46.932531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:46.981425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634586e-f2db-44ae-8ca3-bf6bc6572ee6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.016932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.017631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.017923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.018205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.019387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.020959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.022657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 68/130] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:47.023501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.120123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:47.124279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:47.133944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac1bd-66e7-4260-86c1-d9552174a2d4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.171986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.172863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.173287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.173887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.175243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.177069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.178912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 69/131] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:47 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:47.183708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.200070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ed6f9-a590-4266-bb90-d9166457cb17 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.303611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.304623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.304795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630acf5-1035-4c94-8d27-c81702f15b0a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.305283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.305750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.307134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.308794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.310586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 63/132] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:46 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:47.311569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.353368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.354945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.355238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.355517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/5009c224905e4768b33c681c0f95b0f8/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.357777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, user_id=5009c224905e4768b33c681c0f95b0f8, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.359271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f4c48-a67f-4e3c-8d70-2e8992adcd2e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.379221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 70/133] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:47 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/5009c224905e4768b33c681c0f95b0f8/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:47.382120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9215c9-f82b-4891-8947-58de0e14e97a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.382642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9215c9-f82b-4891-8947-58de0e14e97a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.383155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9215c9-f82b-4891-8947-58de0e14e97a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.386917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 71/134] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:47.388152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9215c9-f82b-4891-8947-58de0e14e97a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.427250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:47.446211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0adb5f-9e8d-4ac5-8c5f-46efdcd06f62 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.487918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.488638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.488923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.489201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.490497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.492375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70cf72-8663-4851-b396-0c333732f8ea None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.494129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 64/135] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:47 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:47.495462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.496048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.496334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.518191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:47.519596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:47.535389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:47.605967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 65/136] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:47.607340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.646100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9215c9-f82b-4891-8947-58de0e14e97a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.718646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a66316-e132-46da-afa1-435c322d218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.719316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66316-e132-46da-afa1-435c322d218b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.719598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66316-e132-46da-afa1-435c322d218b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.719923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66316-e132-46da-afa1-435c322d218b None admin] PATH_INFO: `/v3/projects/b1f8e0cd5804485198d1bc117e18f553` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.727524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a66316-e132-46da-afa1-435c322d218b None admin] RBAC: Authorizing `identity:get_project(project_id=b1f8e0cd5804485198d1bc117e18f553)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.729111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a66316-e132-46da-afa1-435c322d218b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.730918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 72/137] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:47 2022] GET /identity/v3/projects/b1f8e0cd5804485198d1bc117e18f553 => generated 312 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:47.754196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a66316-e132-46da-afa1-435c322d218b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:47.767245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd5c3d-be74-41a1-bef1-7d465ef804aa None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.804247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.804925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.805213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.805491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.807856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, user_id=1ccc32293d63478eb9e68ea2e8c599ba, role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.809763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:47.826496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 66/138] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:47 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/c566cf204ec94aaba0a8466d98161f8d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:47.907070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:47.938610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a66316-e132-46da-afa1-435c322d218b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:47.982094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:47.983113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:47.983393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:47.983668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] PATH_INFO: `/v3/projects/b1f8e0cd5804485198d1bc117e18f553/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:47.985701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1f8e0cd5804485198d1bc117e18f553, user_id=1ccc32293d63478eb9e68ea2e8c599ba, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:47.987246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:48.014935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 73/139] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:47 2022] PUT /identity/v3/projects/b1f8e0cd5804485198d1bc117e18f553/users/1ccc32293d63478eb9e68ea2e8c599ba/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:48.069806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:48.081277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:48.184078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8eecf-79e5-434a-8332-fff5926e9483 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:48.221533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:48.222194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:48.222472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:48.222746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:48.224081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:48.225552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2782f-0be5-4fed-8389-1986b9e4f8c5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:48.227375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 67/140] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:48.234858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:48.350791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4ca35-0bee-4b78-a19b-455b142f4ae7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:48.389395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:48.390031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:48.390310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:48.390580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:48.391332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:48.392787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c25d34-0a07-4d2c-85ba-c33277379d85 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:48.402357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:48.428958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 74/141] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:35:48 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:48.645769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3d2b-dc1b-4eee-9aa2-d7dfb14b4b19 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:48.646489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3d2b-dc1b-4eee-9aa2-d7dfb14b4b19 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:48.647013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3d2b-dc1b-4eee-9aa2-d7dfb14b4b19 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:48.648896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 68/142] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:48.654941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56766-708c-4779-934d-75e13e037dcc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:48.655526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56766-708c-4779-934d-75e13e037dcc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:48.656022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56766-708c-4779-934d-75e13e037dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:48.677135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-13a56766-708c-4779-934d-75e13e037dcc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:48.678754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-13a56766-708c-4779-934d-75e13e037dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:48.770286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 75/143] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:49.041306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bffc5b-715a-4a96-99be-6c580627f341 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.042305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bffc5b-715a-4a96-99be-6c580627f341 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.042834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bffc5b-715a-4a96-99be-6c580627f341 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.044389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 69/144] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:49.049629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.050437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.050955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.080081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:49.081673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:49.157258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 76/145] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:49.159486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bffc5b-715a-4a96-99be-6c580627f341 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:49.246914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:49.275800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bffc5b-715a-4a96-99be-6c580627f341 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:49.319065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:49.319652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.319938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.320215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.334499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:49.339573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:49.340403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:49.341157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 70/146] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:49.348341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:49.410207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:49.472961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842683bb-88e0-4d38-80c3-a61fb39cb058 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:49.509800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:49.510427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.510707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.511001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.511675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:49.511976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:49.513664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:49.526474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 77/147] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:49.532368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:49.641383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e82-425b-4bac-bea3-2ae6c7bf15cd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:49.679081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:49.679728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.680015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.680287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.681439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:49.682937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:49.684846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 71/148] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:49 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:49.690730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:49.783962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:49.789821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:49.818262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7ecb2-59b5-4b08-bf79-9d1827fa8070 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:49.835429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:49.875222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:49.875844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:49.876118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:49.876488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:49.964207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:49.965670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:49.967321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802a80-2fa6-486a-a9a4-68760bc53236 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.032515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.033639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.034206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.034679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.059319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:50.065098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:50.065942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:50.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:50.069394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 72/149] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:50.075007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.145112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 78/150] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:50.155570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:50.161972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.191317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:50.204918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7839-c566-4d86-b6f8-35192aad8eea None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.260895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.261797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.262269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.262733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.264152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:50.264656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:50.266263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:50.276350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 73/151] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:50.282117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.376436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b002f2-129c-4a91-9766-25ed4e45a974 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.433012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d168-8962-43cb-b2d7-81a5e91c17f1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.447521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.448173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.448462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.448761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.450002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:50.451581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:50.453231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 79/152] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:50 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:50.460048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.478077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:50.491823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.492692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.493210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.493591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] PATH_INFO: `/v3/users/26b045e8d8344e6e94db0f2036a418bd` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.495941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] RBAC: Authorizing `identity:get_user(user_id=26b045e8d8344e6e94db0f2036a418bd)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:50.498276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:50.500606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 74/153] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:50 2022] GET /identity/v3/users/26b045e8d8344e6e94db0f2036a418bd => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:50.506904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.555438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:50.575307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea433e-0c60-49b1-bc51-9ec360dea991 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.610207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.611051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.611462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.611792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.612628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:50.613003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:50.614661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154da42d-c955-4443-a71a-907bfaa8b5f7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:50.621377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 80/154] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:50 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=87b7a008126642369351bd284f3e8354 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:50.645302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8c218-1a54-4ed7-801e-58cb8c9126fe None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.645770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8c218-1a54-4ed7-801e-58cb8c9126fe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.646153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8c218-1a54-4ed7-801e-58cb8c9126fe None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.647881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 81/155] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:50.653331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.653723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.655047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.676324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:50.678686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:50.721095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb4747-d740-4297-b03f-927939b400cd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.763543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 82/156] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:50.805220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bea9e-68a1-438b-a110-06176282152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:50.805899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bea9e-68a1-438b-a110-06176282152d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.806184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bea9e-68a1-438b-a110-06176282152d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.806465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bea9e-68a1-438b-a110-06176282152d None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.811623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bea9e-68a1-438b-a110-06176282152d None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:50.813121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bea9e-68a1-438b-a110-06176282152d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:50.815109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 75/157] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:50 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:50.821209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:50.930309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:50.948371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6cf2-b5b7-4edc-9d74-afb9600fd0b1 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.948738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6cf2-b5b7-4edc-9d74-afb9600fd0b1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.948935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6cf2-b5b7-4edc-9d74-afb9600fd0b1 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.950594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 76/158] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:50.959207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e26e2-5c04-4df9-91e8-dc434e8abc59 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:50.964312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:50.964596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:50.964870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:50.999319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:51.000047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:51.000326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:51.000599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/26b045e8d8344e6e94db0f2036a418bd/roles/7612f61be5c745b6927ee4bea9f4eeba` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:51.002700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, user_id=26b045e8d8344e6e94db0f2036a418bd, role_id=7612f61be5c745b6927ee4bea9f4eeba)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:51.004181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:51.007824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:51.009181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:51.029628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 83/159] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:50 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/users/26b045e8d8344e6e94db0f2036a418bd/roles/7612f61be5c745b6927ee4bea9f4eeba => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:35:51.113285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:51.125565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 77/160] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:51.552389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:51.684317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf89f1-6ea0-4546-9a7a-1cb645ed6e81 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:51.710669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:51.734142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:51.734798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:51.735111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:51.735409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:51.743170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:51.751407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-fd1165d5-2159-490d-a9f2-ca00d82f4094 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:51.756777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:51.757659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:51.758409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 84/161] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:51.759832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f750-7c93-4ce4-9b8d-ba0cb09e4647 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:51.759976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f750-7c93-4ce4-9b8d-ba0cb09e4647 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:51.760155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f750-7c93-4ce4-9b8d-ba0cb09e4647 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:51.764444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 85/162] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:51.767908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1f750-7c93-4ce4-9b8d-ba0cb09e4647 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:51.918224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893287b3-b6d3-4c83-9656-907a0573fd7d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:51.949006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1f750-7c93-4ce4-9b8d-ba0cb09e4647 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:51.968072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:51.968734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:51.969077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:51.969354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:51.980568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-b98ec69f-8e1f-47e7-bfe5-7bf5b0331771 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:51.988731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:51.989673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:51.990880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 78/163] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:35:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:51.991515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94285074-ddec-4617-bffe-5908e762e348 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:51.991802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94285074-ddec-4617-bffe-5908e762e348 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:51.992047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94285074-ddec-4617-bffe-5908e762e348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.029488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-94285074-ddec-4617-bffe-5908e762e348 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:52.030035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.030665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.030764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-94285074-ddec-4617-bffe-5908e762e348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:52.030955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.031225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.031942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:52.032244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:52.033822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:52.054503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 86/164] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:52.055209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.108309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:52.142072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 79/165] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:52.142957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94285074-ddec-4617-bffe-5908e762e348 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.143511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:52.214698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d2b8-8515-40af-98dd-5ac170feb779 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:52.301728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.302931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.304032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.304646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.314827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:52.316130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:52.318353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08706-61fc-4d4b-b658-2320a2800c91 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:52.328477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94285074-ddec-4617-bffe-5908e762e348 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:52.341641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 87/166] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:52.343259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3172c7-d081-4093-8fd4-a4f8f7806e90 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.343755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3172c7-d081-4093-8fd4-a4f8f7806e90 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.344369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3172c7-d081-4093-8fd4-a4f8f7806e90 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.351594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 88/167] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:52.352718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3172c7-d081-4093-8fd4-a4f8f7806e90 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.416402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.417431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.418086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.419151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] PATH_INFO: `/v3/users/5009c224905e4768b33c681c0f95b0f8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.421308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] RBAC: Authorizing `identity:get_user(user_id=5009c224905e4768b33c681c0f95b0f8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:52.432314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4e8ef-c28f-47b0-b387-6d3fd91044f2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:52.434928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 80/168] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/users/5009c224905e4768b33c681c0f95b0f8 => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:52.437010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.437649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.439312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.478875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:52.481117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:52.585035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:52.596643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3172c7-d081-4093-8fd4-a4f8f7806e90 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:52.623133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:52.627111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 81/169] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:52.627998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.652830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.653569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.653840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.654136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.657163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:52.659009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:52.660938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 89/170] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:52.661858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.812331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d252bbb-423e-4d65-9b3b-f00beed532d9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:52.818926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cff1-7993-4bae-87f2-7aa39ec6e5aa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:52.855329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.856202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.856613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.857043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.858648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:52.859524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:52.859975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:52.860408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:52.876626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:52.881708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:52.882556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:52.883365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:52.890163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 90/171] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:52.891251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:52.919260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:52.932343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:52.934102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:52.976681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.035831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 82/172] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:53.036973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.062930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620bd32-2330-41ca-9b35-c2481619dfe2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.122403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.132165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.132979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.133417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.133818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.150733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325e0e-2618-435e-9e42-42a8cf5105bf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.185931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.186778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.187236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.187638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.188835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:53.189569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.191614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.202826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 83/173] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.203788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.207949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.209050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89a0ce-0e3f-419a-a826-0a502f1656e4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.271253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.314337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b185f-17aa-4510-92b8-45c446b08603 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.321186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 91/174] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:53.322851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac6b5-3887-488b-954d-98bb366bedff None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.323378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac6b5-3887-488b-954d-98bb366bedff None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.323845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac6b5-3887-488b-954d-98bb366bedff None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.325314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 92/175] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:53.326238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ac6b5-3887-488b-954d-98bb366bedff None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.363726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.364421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.364699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.364971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.366105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.367774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.369420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 84/176] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.374701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.386964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.447244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ac6b5-3887-488b-954d-98bb366bedff None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.491744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9570c-4590-48d4-b7e7-43b8feff0a22 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.493145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.494028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.494305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.494573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.508636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:53.514738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:53.515643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:53.516639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 93/177] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:53.517285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.529412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.530051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.530624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.530624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] PATH_INFO: `/v3/users/1ccc32293d63478eb9e68ea2e8c599ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.531253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.531723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccc32293d63478eb9e68ea2e8c599ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.533194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf51aec-538e-4497-b3a6-e690e063de55 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.538857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 85/178] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/users/1ccc32293d63478eb9e68ea2e8c599ba => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.538857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.538857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.538857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.555036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:53.556357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:53.627126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e6392-d797-4060-aec5-d4f3d7ce3f29 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.638994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 86/179] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:53.638994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.649532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.664545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.665401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.665816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.666227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.667560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.669223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dbf36-9eb3-4a43-9fb0-c32b27a15a78 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.671130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 94/180] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.672503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407a162-78ce-4671-9a57-ca9f10bfda55 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.672937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407a162-78ce-4671-9a57-ca9f10bfda55 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.673348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407a162-78ce-4671-9a57-ca9f10bfda55 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.674708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 95/181] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:53.679573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407a162-78ce-4671-9a57-ca9f10bfda55 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.761744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.774898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2186880-f9e3-4c98-b364-592b1a18fd54 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.798382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407a162-78ce-4671-9a57-ca9f10bfda55 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.812938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.813857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.814337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.814850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.815817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=5009c224905e4768b33c681c0f95b0f8) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:53.816360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.818038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.824832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 87/182] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/role_assignments?user.id=5009c224905e4768b33c681c0f95b0f8&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.826004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.845710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.846476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.846784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.847481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:53.848225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:53.848582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:53.850060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:53.864161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 96/183] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:53.868979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:53.909935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:53.946101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242524-15a1-48a5-ab05-94560f7eacdb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:53.984124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:53.984750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:53.985026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:53.985293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.004356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045b95e-e758-401f-9a74-9e8bd409dc8c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.042198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.042827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.043104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.043371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.044091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:54.044392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.045829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38013c-6ca6-41b6-aedb-2774307ffed7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.053887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 97/184] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:54.053887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.054265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.054421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.061721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.068318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.069917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53c48f-b989-4a78-801e-949e52e3c5bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.078449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:54.080805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:54.158787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.160613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 98/185] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:54.163771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.194731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 88/186] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:54.196376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199142dc-d157-45c0-aba7-8a9034a3a848 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.196660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199142dc-d157-45c0-aba7-8a9034a3a848 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.196928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199142dc-d157-45c0-aba7-8a9034a3a848 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.198265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 89/187] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:54.202425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199142dc-d157-45c0-aba7-8a9034a3a848 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.290596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617d73f-dc90-4b6a-9223-10698191f795 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.327363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.333203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.333833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.334213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.334489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.338466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199142dc-d157-45c0-aba7-8a9034a3a848 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.348154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.349683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.351363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 99/188] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:54.352114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.374543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.375207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.375486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.375755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.376861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.378334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509d85d-a9b0-4984-8fff-c20f62d00f06 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.380011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 90/189] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:54.384736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.385598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.386051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.409136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:54.410596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:54.450956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.457014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b62c2-ff0a-45e0-9b3f-ef9f70c07024 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.484308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 91/190] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:54.485147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.491698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.492325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.492597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.492868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] PATH_INFO: `/v3/projects/b1f8e0cd5804485198d1bc117e18f553` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.493975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] RBAC: Authorizing `identity:get_project(project_id=b1f8e0cd5804485198d1bc117e18f553)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.495560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.501857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 100/191] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/projects/b1f8e0cd5804485198d1bc117e18f553 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:54.501857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.553658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.597257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe44ac2-c31c-4f94-b031-f97ec884e349 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.598827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df0cab-b374-48fd-8fe5-9d31d9ca85a8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.632973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.633584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.633863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.634133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.635740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.636401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.636692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.636962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.637992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.639073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.662251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 101/192] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:35:54 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:35:54.662938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.689939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.734660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:54.738192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:54.782881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b1cc-dcfc-4d8b-a6b4-b727a70ad90c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.800064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.826178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:54.826910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:54.828368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:54.828368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:54.840442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:54.845243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:54.846187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 102/193] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:54.851834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.863011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 92/194] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:54.863864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:54.957267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:54.968986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba6b10-fca6-4023-b245-e7a39748b721 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:54.989913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2716-1208-4957-90ed-ad6c19584a4e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.009744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.011016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.011498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.011945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.012858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=b1f8e0cd5804485198d1bc117e18f553, user.id=1ccc32293d63478eb9e68ea2e8c599ba) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:55.013754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.015682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.024066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 103/195] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/role_assignments?user.id=1ccc32293d63478eb9e68ea2e8c599ba&scope.project.id=b1f8e0cd5804485198d1bc117e18f553&role.id=87b7a008126642369351bd284f3e8354 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.025114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.040326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.042066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.042698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.044850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.066253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:55.073854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:55.074823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:35:55.075606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:55.080004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 93/196] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:55.082747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.114473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.233911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ef45-5bae-46ff-9452-41b18e097b84 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.240653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e76ae5-e25f-41c8-b184-8bd586a0151d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.282378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.283238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.283614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.283982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.284944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:55.285345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.286328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.287110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.287345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.287763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.288192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.289541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.291268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.293077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 94/197] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.294061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.296652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 104/198] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.297232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.299769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.410060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcef3-338f-4897-8575-13eff86d21fa None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.429023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b8a2-269f-4279-aa98-35f9375f2a41 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.430962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.461503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.462128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.462404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.462674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.463390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] RBAC: Adding query filter params (name=reader) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:55.463691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.465082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.475611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.476233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.476509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.476779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.477511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=75fc992521854ebfab064c823bd457e9) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:55.477815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.478921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 95/199] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.479288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.479808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.487580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 105/200] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/role_assignments?user.id=75fc992521854ebfab064c823bd457e9&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.488403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.557598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.602291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f923a-7e8d-4b50-9308-b37dab2829da None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.631415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a1943-53e9-488d-a922-8a4afea48956 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.663544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.664212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.664486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.664752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.670787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.671451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.671611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.671757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.671987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] PATH_INFO: `/v3/users/26b045e8d8344e6e94db0f2036a418bd` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.672815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.673190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] RBAC: Authorizing `identity:get_user(user_id=26b045e8d8344e6e94db0f2036a418bd)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.674719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:55.674921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 96/201] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.676602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 106/202] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/users/26b045e8d8344e6e94db0f2036a418bd => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:55.685209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.685490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:55.726131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.813744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723ae07-b806-4922-ab96-6c24eb5579cc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.852311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b3153-1f33-4eb9-b429-04aa41ad3ff7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:55.887004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.887626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.890258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.890543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.913022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:55.930234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:55.930884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:55.931165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:55.931441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:55.974452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:55.976484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.011304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:56.079012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:56.080436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.120488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 107/203] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:35:56.125995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:56.270909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 97/204] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:56.276566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:56.278645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3406-d4a1-4af9-b8fd-d7baf17d2230 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:56.320280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:56.340192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:56.353309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:56.354067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.354346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.354722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:56.359520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:56.361003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.362689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 108/205] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:56 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:56.369537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:56.434732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0461798-63d5-4cfb-8dc5-1346d764488f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:56.503839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:56.504559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.504845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.505120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:56.506286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:56.507786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.507904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c295-00f4-4e12-a3ce-5a20b6f46d3a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:56.509753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 98/206] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:56 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:56.515409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:56.543515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:56.544160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.544406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.544677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:56.545409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=75fc992521854ebfab064c823bd457e9) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:56.545714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:56.547180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266146-4d7c-4784-a2d1-3ef8a4e6665c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.552996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 109/207] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:56 2022] GET /identity/v3/role_assignments?user.id=75fc992521854ebfab064c823bd457e9&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:56.639458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:56.646793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c2ae1-ee81-40dd-b6db-b4c042267736 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:56.683564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:56.684190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.684470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.684742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:56.685482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab, user.id=26b045e8d8344e6e94db0f2036a418bd) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:56.685790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:56.687312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75377ead-88dc-4d46-b70a-7c2944e7bd83 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:56.694039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 99/208] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:56 2022] GET /identity/v3/role_assignments?user.id=26b045e8d8344e6e94db0f2036a418bd&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:56.975215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce8cd-a424-4984-af33-b099e514ae07 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.976086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce8cd-a424-4984-af33-b099e514ae07 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.976359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce8cd-a424-4984-af33-b099e514ae07 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:56.977950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 110/209] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:56.988332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:56.988947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:56.989419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.010323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:57.011612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:57.038899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb57f64-3db8-42ed-a7a3-b8fb00a35fe6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.039520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb57f64-3db8-42ed-a7a3-b8fb00a35fe6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.040025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb57f64-3db8-42ed-a7a3-b8fb00a35fe6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.041817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 111/210] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:57.047533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.048079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.048571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.072412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:57.073956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:57.097966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 100/211] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:57.143625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:57.154830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 112/212] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:57.375344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:57.426704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:57.474059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:57.488666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e77641-9257-4415-8213-af52dfd3cd58 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:57.526547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:57.527690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.528176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.528853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.530163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:57.531282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2792-f85e-4d48-8260-8fc385554532 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:57.532163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:57.534489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 101/213] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:35:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:57.543534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:57.565637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:57.566278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.566554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.566844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.577077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:57.581874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:57.582856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:57.589452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 113/214] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:57.591646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:57.667353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:57.669061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a708-7cc9-4ac7-8feb-f14daf82b786 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:57.712933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:57.713604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.713889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.714301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.715000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:57.717094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4e814-1e19-4b2b-a20e-17bb99a86941 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:57.724334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081cb73-e0fe-4edc-b04f-ee6e8292f994 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:57.754692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 102/215] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:35:57 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:35:57.771480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:57.772377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.772824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.773290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.774176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:57.774624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:57.776383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:57.779978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7939816-f912-48c8-a46a-25b5fb15e4c4 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.780258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7939816-f912-48c8-a46a-25b5fb15e4c4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.780627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7939816-f912-48c8-a46a-25b5fb15e4c4 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.782449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 103/216] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:35:57.792939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.792939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.792939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.793619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 114/217] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:35:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:57.795576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:57.811974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:57.813397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:57.825381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:57.906835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 104/218] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:35:57.940250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddde518-0106-40d5-ad33-8aa93423d122 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:57.964228 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:57.976936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:57.978285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:57.978753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:57.979324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] PATH_INFO: `/v3/users/969d60d6f7804c009fd405177d1f7e51` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:57.994560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] RBAC: Authorizing `identity:get_user(user_id=969d60d6f7804c009fd405177d1f7e51)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:57.996398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:57.998269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 115/219] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:57 2022] GET /identity/v3/users/969d60d6f7804c009fd405177d1f7e51 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:58.004283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.118156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9442d2c-334a-473f-8037-406946a9ec4d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.153440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.154393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.154915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.155374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.170595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:58.178620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.233658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:58.235514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:58.301941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4e51-0b0d-4d11-8aa0-4b74a459c99c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.338874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.339500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.339810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.340092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.351467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:58.357028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:58.358223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:35:58.359032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:35:58.362906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 116/220] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:58.364554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.373780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 105/221] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:35:58.381547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.409220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:58.491437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:58.534777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f615cdc-6d66-4319-b41b-c86cd42bdce4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.591498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfc1b0-d0ef-4823-89f0-0cf5e10b6bff None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.599579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.600208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.600482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.600752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.601414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:58.601818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:58.603320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:58.616623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 117/222] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:58.622725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.627476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440d298-3e12-458a-876c-37e440129997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.628062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440d298-3e12-458a-876c-37e440129997 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.628341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440d298-3e12-458a-876c-37e440129997 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.628616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440d298-3e12-458a-876c-37e440129997 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.629824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440d298-3e12-458a-876c-37e440129997 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:58.631385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440d298-3e12-458a-876c-37e440129997 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:58.633379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 106/223] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:58.638880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440d298-3e12-458a-876c-37e440129997 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.761595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:58.770359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f589d-7d97-4087-ba8d-4d67bc06fb5d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.782650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440d298-3e12-458a-876c-37e440129997 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.800353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:58.810500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.811438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.811721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.811994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.813262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:58.815339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e8c19-a51d-4e1f-963b-2e99f7543e64 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:58.817989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 118/224] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:58.819311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f162-7222-423b-9a24-87935fb11245 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.819590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f162-7222-423b-9a24-87935fb11245 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.819860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f162-7222-423b-9a24-87935fb11245 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.821052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 119/225] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:35:58.824496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f162-7222-423b-9a24-87935fb11245 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:58.831920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.832630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.832965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.833294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.834171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=969d60d6f7804c009fd405177d1f7e51) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:58.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:58.836629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a63d3-f96b-4b41-b188-21d5bf4eb20c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:58.846179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 107/226] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/role_assignments?user.id=969d60d6f7804c009fd405177d1f7e51&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=87b7a008126642369351bd284f3e8354 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:35:58.847411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.847690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:58.847961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:58.896132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:35:58.897272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:35:58.954330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f162-7222-423b-9a24-87935fb11245 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:58.998776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:58.999556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:58.999836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:59.000105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:59.001456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:59.003529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:59.010439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 120/227] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:35:58 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:59.011891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:59.052680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:59.081411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 108/228] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:35:59.133575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6a013-d940-491c-8ae7-d82729f4cf01 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:59.169205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:59.169794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:59.170070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:59.170384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/75fc992521854ebfab064c823bd457e9/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:59.172404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, user_id=75fc992521854ebfab064c823bd457e9, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:59.173856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:59.191315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 121/229] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:35:59 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/75fc992521854ebfab064c823bd457e9/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:35:59.441996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:59.565776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf329f-f758-4bad-9615-db90c5c63502 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:59.576529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:59.607427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:59.608066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:59.608342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:59.608613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:59.619052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:35:59.623897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:35:59.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:35:59.627880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 109/230] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:35:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:35:59.634830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:59.748823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:35:59.762171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a23e6-ca19-43c2-8d43-c0d9e7fe3760 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:59.807342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:59.807958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:59.808234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:59.808506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:59.809171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:35:59.809475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:59.810873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:59.819217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 122/231] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:35:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:35:59.824870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:35:59.946033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52bb07-505d-4e5c-9a17-64ec5b6cb07a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:35:59.988591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:35:59.989299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:35:59.989579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:35:59.989846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:35:59.994418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:35:59.995941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:35:59.997673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 110/232] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:35:59 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:00.003260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.123437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a08f-4b6b-42b2-b625-3cae3f4539f8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:00.157696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:00.158749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.159252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.159738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.161131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:00.162768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235244ca-d184-49ef-926e-99b7a16a9e62 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:00.164953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 123/233] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:00 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:00.171045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.175926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6ab0f-32d6-472b-8ceb-5f46abc98d1a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.176251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6ab0f-32d6-472b-8ceb-5f46abc98d1a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.176530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6ab0f-32d6-472b-8ceb-5f46abc98d1a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.177985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 124/234] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:00.192056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2505-1a28-4c29-bfd5-5cf7a398709b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.192341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2505-1a28-4c29-bfd5-5cf7a398709b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.192615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2505-1a28-4c29-bfd5-5cf7a398709b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.216702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:00.218630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-906c2505-1a28-4c29-bfd5-5cf7a398709b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:00.219855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-906c2505-1a28-4c29-bfd5-5cf7a398709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:00.239299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:00.323182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 125/235] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:00.323583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6af8a4-b5e8-4987-9731-362f00bef4ef None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:00.324449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c7bc-ad16-459e-878a-945c34dd0057 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.324734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c7bc-ad16-459e-878a-945c34dd0057 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.325008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c7bc-ad16-459e-878a-945c34dd0057 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.327946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 126/236] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:00.342955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117f34-573d-45e7-a527-364efbe66129 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.343271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117f34-573d-45e7-a527-364efbe66129 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.343554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117f34-573d-45e7-a527-364efbe66129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.376829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:00.377532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.377808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.378084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/75fc992521854ebfab064c823bd457e9/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.380320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-de117f34-573d-45e7-a527-364efbe66129 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:00.382074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-de117f34-573d-45e7-a527-364efbe66129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:00.383516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, user_id=75fc992521854ebfab064c823bd457e9, role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:00.385057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:00.406653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 111/237] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:00 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/75fc992521854ebfab064c823bd457e9/roles/c566cf204ec94aaba0a8466d98161f8d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:36:00.471207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 127/238] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:00.645696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.748948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:00.757360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc31659-1a48-4e93-be90-9e4d91e3b16d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:00.761066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de117f34-573d-45e7-a527-364efbe66129 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.799261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:00.800009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.800395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.800667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.811599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:00.817152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:00.818010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:00.818833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 112/239] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:00.826111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.864105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:00.894093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de117f34-573d-45e7-a527-364efbe66129 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:00.938204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9023f-abca-4870-82f0-d6ebeb375edc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:00.940070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:00.940839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.942143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.942526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.943788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:00.945346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:00.947895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 128/240] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:00.953341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:00.978753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:00.979812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:00.980257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:00.980695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:00.981543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:00.982470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:00.984229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:00.998184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 113/241] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:01.000103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:01.020076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:01.057339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fe26-38ac-4e5b-9cc4-0b1e7dcc947e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:01.092602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:01.093457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.093943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.094411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.095321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:01.097010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:01.110178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2685dc-a6ff-4ae9-9687-0e432e1491ba None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:01.132384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 129/242] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:00 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:36:01.148959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:01.149946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.150398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.150902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] PATH_INFO: `/v3/users/15f30caf99c84773b9735236f0e812c0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.167435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] RBAC: Authorizing `identity:get_user(user_id=15f30caf99c84773b9735236f0e812c0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:01.169160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:01.171211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 114/243] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:00 2022] GET /identity/v3/users/15f30caf99c84773b9735236f0e812c0 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:01.177554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:01.188178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:01.299773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dceb4c-1a2b-4cbc-ab54-e8965388eda6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:01.335877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:01.336498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.336776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.337043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.426307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:01.427969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814c008-2930-43f2-bc61-f4b123bde3da None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:01.437825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:01.542558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 130/244] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:01.543976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:01.603412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dac4-4a75-4f03-900a-7b0142a11de3 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.603733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dac4-4a75-4f03-900a-7b0142a11de3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.603954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dac4-4a75-4f03-900a-7b0142a11de3 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.605608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 131/245] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:01.611239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135c0d0-23e6-4a08-ad08-ae860a877df6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.611607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135c0d0-23e6-4a08-ad08-ae860a877df6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.611880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135c0d0-23e6-4a08-ad08-ae860a877df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.633080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:01.634508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a135c0d0-23e6-4a08-ad08-ae860a877df6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:01.635878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a135c0d0-23e6-4a08-ad08-ae860a877df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:01.667033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b448b-b69d-411d-a9d6-9c2ac9ac4e6e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:01.714785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:01.715454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.715734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.716009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.717155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:01.718627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:01.720584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 115/246] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:01 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:01.726235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:01.736815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 132/247] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:01.761953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0bcf6-8bb1-49ec-8429-679ae2a3b8b0 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.762317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0bcf6-8bb1-49ec-8429-679ae2a3b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.762601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0bcf6-8bb1-49ec-8429-679ae2a3b8b0 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.763977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 133/248] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:01.772088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.772380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.772621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.795033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:01.796139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:01.838285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:01.857732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a69b1-32d7-4a62-ba52-cc2426eaa1ee None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:01.893741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 134/249] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:01.907938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:01.908830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:01.909332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:01.909867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:01.910849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=15f30caf99c84773b9735236f0e812c0) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:01.911302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:01.912941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:01.919578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 116/250] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:01 2022] GET /identity/v3/role_assignments?user.id=15f30caf99c84773b9735236f0e812c0&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:02.150971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.277505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c468894-63ab-448e-8b50-bb39e52dfcfe None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:02.326086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.328342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:02.329291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:02.329821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:02.330289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:02.344282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:02.350748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:02.361880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:02.362745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:02.363589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:02.364182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 135/251] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:02.372197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.438275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:02.575728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccf187-aa4a-4630-b39c-a3712d944336 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:02.588921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d92b1-feb0-4405-a190-3c85a2880ee3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:02.610749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:02.611704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:02.612146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:02.612549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:02.613345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:02.613819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:02.615376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:02.624044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 136/252] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:02.632467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.674798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:02.676097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:02.676579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:02.677013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:02.707909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:02.718044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:02.719093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:02.720226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 117/253] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:02.727710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.761948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d149-0d4b-4430-a5af-a080f0e6f684 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:02.776251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:02.800024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:02.800973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:02.801419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:02.802384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:02.803822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:02.805446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:02.807653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 137/254] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:02.815460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:02.885981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:02.949715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342bf8-96db-4b6b-8cb8-8a4f47665c31 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.016783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd6d2d-a3a1-429d-9248-d91ee8cdcff6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.018368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.019027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.019309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.019582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.020331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:03.020636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.022054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.045172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 118/255] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:03.050461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:03.119411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.120035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.120506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.120785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.176823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:03.249025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.255756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225749-d7bc-470f-82a2-313795d0c791 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.258630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba84889-3d48-4e23-b04d-55e4e1be20b1 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.319327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:03.335500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.336159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.336435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.336708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] PATH_INFO: `/v3/users/969d60d6f7804c009fd405177d1f7e51` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.337883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] RBAC: Authorizing `identity:get_user(user_id=969d60d6f7804c009fd405177d1f7e51)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.339577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab33f2-a535-49a6-a827-c31a0bf37519 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.341494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 119/256] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:03 2022] GET /identity/v3/users/969d60d6f7804c009fd405177d1f7e51 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:03.342631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ccc89-12e9-4edf-a992-be1dafc6b49c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.342933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ccc89-12e9-4edf-a992-be1dafc6b49c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.343203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ccc89-12e9-4edf-a992-be1dafc6b49c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.351252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 120/257] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:03.352096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ccc89-12e9-4edf-a992-be1dafc6b49c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:03.410248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 138/258] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:03.416668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.417391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.418097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.440923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:03.442907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:03.516896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:03.543006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 139/259] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:03.544556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:03.589462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ccc89-12e9-4edf-a992-be1dafc6b49c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.627120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.627756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.628031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.628299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.629705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.631494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.633959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 121/260] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:03 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:03.639352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:03.710515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:03.755298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8d2c2-6765-48e2-beb3-05915713a4ef None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.758590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f3738-5499-496b-9c8a-332cb65b9a61 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:03.812960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.813628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.813904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.814172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/969d60d6f7804c009fd405177d1f7e51/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.815507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:03.816127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:03.816395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:03.816707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:03.818220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.819698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.822077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, user_id=969d60d6f7804c009fd405177d1f7e51, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:03.823552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:03.825993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 140/261] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:03 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:03.827593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:03.850556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 122/262] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:03 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/969d60d6f7804c009fd405177d1f7e51/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:36:03.916468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:03.967782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb78-f3ca-4702-a703-269c7923e754 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:04.010266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:04.010909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.011185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.011453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.012184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=75fc992521854ebfab064c823bd457e9) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:04.012487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:04.013958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:04.020900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 141/263] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:03 2022] GET /identity/v3/role_assignments?user.id=75fc992521854ebfab064c823bd457e9&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:04.231564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:04.370874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d2238-2b18-4c17-9403-1e21c76cd529 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:04.408008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:04.419584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:04.420367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.420850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.421191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.435881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-61b3e64c-9ce1-43ef-955b-8dae9f3dc24d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:04.445289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:04.446168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:04.447365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 123/264] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:04.454177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:04.545971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ca65-ad1a-420e-8445-3d5d1062544e None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.546264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ca65-ad1a-420e-8445-3d5d1062544e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.546646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ca65-ad1a-420e-8445-3d5d1062544e None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.550234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 124/265] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:04.554514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cfc4-3a16-454c-b95b-ba6e88f39eae None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.554947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cfc4-3a16-454c-b95b-ba6e88f39eae None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.555328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cfc4-3a16-454c-b95b-ba6e88f39eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.571373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef517-a3c7-4b58-b3f2-91ac17a60e8b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:04.591188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:04.603539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ea28cfc4-3a16-454c-b95b-ba6e88f39eae None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:04.604773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ea28cfc4-3a16-454c-b95b-ba6e88f39eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:04.613156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:04.613799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.614074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.614340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.615027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:04.615326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:04.616702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:04.625834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 142/266] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:04.632311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:04.745950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 125/267] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:04.752264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428227-a29f-45a9-b3cb-e9ac213ad57b None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.752584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428227-a29f-45a9-b3cb-e9ac213ad57b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428227-a29f-45a9-b3cb-e9ac213ad57b None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.754875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 126/268] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:04.761119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704213fc-b25d-41ea-8ee3-228f426820c1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.761464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704213fc-b25d-41ea-8ee3-228f426820c1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.761749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704213fc-b25d-41ea-8ee3-228f426820c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.788107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd88af-9f4d-46bb-bf15-a4f45fa8f550 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:04.802048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-704213fc-b25d-41ea-8ee3-228f426820c1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:04.803689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-704213fc-b25d-41ea-8ee3-228f426820c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:04.826507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:04.827254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.827532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.827797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] PATH_INFO: `/v3/users/75fc992521854ebfab064c823bd457e9` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.828974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] RBAC: Authorizing `identity:get_user(user_id=75fc992521854ebfab064c823bd457e9)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:04.830580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eb813-a983-4b1a-8da7-f3bba6ef91b6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:04.835380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 143/269] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:04 2022] GET /identity/v3/users/75fc992521854ebfab064c823bd457e9 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:04.835380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c024b0-9e24-4382-847d-2a12b2c2e532 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.835380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c024b0-9e24-4382-847d-2a12b2c2e532 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.835380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c024b0-9e24-4382-847d-2a12b2c2e532 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.835380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 144/270] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:04.843384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c024b0-9e24-4382-847d-2a12b2c2e532 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:04.893273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 127/271] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:04.894698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:04.895036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:04.895287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:04.923350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:04.931469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:04.939949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.014087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.045444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c024b0-9e24-4382-847d-2a12b2c2e532 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.060602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 128/272] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:05.061436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.107908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.108872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.109525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.110785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.213007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7b626-bec0-42be-a6c1-e5f281a780fc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.239627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.242161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.260980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.261733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.262080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.262419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.273949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:05.278978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:05.279821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:05.280628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:05.288061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 129/273] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:05.289210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.299461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.336709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.355032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 145/274] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:05.360031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.395548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda69d-5ca1-4e6d-9a2f-9859752bdaa8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.430916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.431551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.431835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.432110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.433386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.435473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.437433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 130/275] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:05.439164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.472002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1576ae-b67d-4b3d-8177-f9aed43b289c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.507676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.508281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.508596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.508819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.509469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:05.509767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.511162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.522869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 146/276] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:05.522869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.534206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.547607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba930-fdd2-4e8e-9f21-e836b2062098 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.549337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.586032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.587001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.587425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.587696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.598039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:05.602781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:05.603678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:05.604476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 131/277] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:05.605123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.635259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f103f3-6b51-4fe2-8b8e-ad95e37593c6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.671664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.672331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.672613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.672889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.674103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.675601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.682070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 147/278] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:05.682070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.715113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85374a-a836-4161-a666-8681fa253a43 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.750710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.751377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.751623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.751896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.752552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.754055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.771223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.777392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 132/279] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:05 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:05.777392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.808961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5991-4766-4be6-b5de-eea68ad9a90b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.809675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:05.863750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.864446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.864730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.865046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] PATH_INFO: `/v3/users/15f30caf99c84773b9735236f0e812c0` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.866394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] RBAC: Authorizing `identity:get_user(user_id=15f30caf99c84773b9735236f0e812c0)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.868707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.876375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 148/280] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/users/15f30caf99c84773b9735236f0e812c0 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:05.877330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:05.901013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfeb33d-d030-4eb5-b11e-681b2c60f57f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:05.939148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:05.939811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:05.940071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:05.940309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:05.940987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:05.941434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:05.942940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:05.956296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 133/281] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:05.957453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:06.004943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312b985-dd5c-4168-8b77-0e8d203ace85 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:06.054279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:06.054925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.055201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.055558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.056190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=75fc992521854ebfab064c823bd457e9) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:06.057221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:06.057940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:06.064530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 149/282] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/role_assignments?user.id=75fc992521854ebfab064c823bd457e9&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:06.065178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:06.070934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:06.081649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:06.102333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba94df-ee63-42e7-b39e-a4141b53a971 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:06.148217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:06.148843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.149363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.149363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.150662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:06.152157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:06.153978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 134/283] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:05 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:06.159840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:06.203640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8dd0-1d00-431a-b625-f76578f31cde None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:06.243271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:06.243925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.244202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.244471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] PATH_INFO: `/v3/users/48f8d350f36349f6bb990b27d2895f94` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.260211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] RBAC: Authorizing `identity:get_user(user_id=48f8d350f36349f6bb990b27d2895f94)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:06.262560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:06.263806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 150/284] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:06 2022] GET /identity/v3/users/48f8d350f36349f6bb990b27d2895f94 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:06.270012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:06.277703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970959e-f823-46fe-a5fe-634d7a632bea None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:06.313451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:06.314126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.314402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.314674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/15f30caf99c84773b9735236f0e812c0/roles/7612f61be5c745b6927ee4bea9f4eeba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.316982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, user_id=15f30caf99c84773b9735236f0e812c0, role_id=7612f61be5c745b6927ee4bea9f4eeba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:06.318231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a02a8b-c9cd-436e-8630-3801e86b2ad5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:06.332896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 135/285] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:06 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/users/15f30caf99c84773b9735236f0e812c0/roles/7612f61be5c745b6927ee4bea9f4eeba => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:06.354673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:06.433593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc7a60-ea11-4c66-bf40-a4dbc4eaa85e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:06.493151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:06.493767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.494053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.496341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.496341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, user.id=48f8d350f36349f6bb990b27d2895f94, scope.system=all) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:06.496341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:06.496796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e42d-9a0e-453c-91b2-1589a3aa5267 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:06.502946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 151/286] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Sep 3 03:36:06 2022] GET /identity/v3/role_assignments?user.id=48f8d350f36349f6bb990b27d2895f94&scope.system=all&role.id=87b7a008126642369351bd284f3e8354 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:06.517824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c897d-9465-4f88-9f10-bf8eb692cf72 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.518124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c897d-9465-4f88-9f10-bf8eb692cf72 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.518368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c897d-9465-4f88-9f10-bf8eb692cf72 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.519906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 136/287] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:06.525650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:06.525938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:06.526214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:06.548692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:06.549904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:06.649539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 152/288] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:06.967666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c897d-9465-4f88-9f10-bf8eb692cf72 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:06.981109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:07.100530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c897d-9465-4f88-9f10-bf8eb692cf72 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:07.148413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:07.149049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:07.149321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:07.149585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:07.159452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:07.164848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:07.165805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:07.166996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 137/289] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:07.172550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:07.250398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:07.303367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b9173-91d7-4e29-8253-749554336de7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:07.353911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:07.354520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:07.354792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:07.355078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:07.355740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:07.356044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:07.357718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:07.368450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 153/290] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:07.374009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:07.452393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:07.486068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35eb4-f147-4ebb-b269-4c49b970ddcd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:07.527560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:07.528115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:07.528388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:07.528656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] PATH_INFO: `/v3/users/969d60d6f7804c009fd405177d1f7e51` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:07.529783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] RBAC: Authorizing `identity:get_user(user_id=969d60d6f7804c009fd405177d1f7e51)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:07.531280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:07.532918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 138/291] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:07 2022] GET /identity/v3/users/969d60d6f7804c009fd405177d1f7e51 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:07.538824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:07.649197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b344-2de2-4589-a353-807f7f95e678 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:07.673089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:07.689302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:07.689916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:07.690195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:07.690464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:07.780976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:07.782349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:07.896087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 154/292] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:07.901695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:08.020997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4d489-0918-4918-a65b-66486111a5df None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:08.062454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:08.063030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.063304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.063573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.067480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:08.068940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaf07b-4335-4a92-b1b9-b898b70e46bd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:08.070696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 139/293] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:07 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:08.076707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:08.103881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:08.204520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf462-04ed-45f8-96a0-54fd5c0d0fa5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:08.258865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:08.259543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.259821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.260090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.260831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=969d60d6f7804c009fd405177d1f7e51) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:08.261134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:08.262652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dc1a5-bd7a-4882-9f47-a86b9cab1832 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:08.269237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 155/294] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:08 2022] GET /identity/v3/role_assignments?user.id=969d60d6f7804c009fd405177d1f7e51&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=87b7a008126642369351bd284f3e8354 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:08.606379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc81f-17b3-4e4c-92ac-331fec53abc5 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.606658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc81f-17b3-4e4c-92ac-331fec53abc5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.607263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc81f-17b3-4e4c-92ac-331fec53abc5 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.608864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 140/295] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:08.614077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6992-c0e4-4aea-a042-4c82bcb55f33 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.614517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6992-c0e4-4aea-a042-4c82bcb55f33 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.614686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6992-c0e4-4aea-a042-4c82bcb55f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.636377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-14bd6992-c0e4-4aea-a042-4c82bcb55f33 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:08.637900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-14bd6992-c0e4-4aea-a042-4c82bcb55f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:08.712833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 156/296] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:08.761326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe11-f713-49c9-a58e-204527a4fb55 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.761630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe11-f713-49c9-a58e-204527a4fb55 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.761868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe11-f713-49c9-a58e-204527a4fb55 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.763573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 141/297] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:08.768554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5f94b-a2a6-480d-ba2e-4034778ae149 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.769069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5f94b-a2a6-480d-ba2e-4034778ae149 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.769604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5f94b-a2a6-480d-ba2e-4034778ae149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.791339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c8b5f94b-a2a6-480d-ba2e-4034778ae149 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:08.792851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c8b5f94b-a2a6-480d-ba2e-4034778ae149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:08.809536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:08.823183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f187-a507-4559-a33a-350f7f16c292 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.823467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f187-a507-4559-a33a-350f7f16c292 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.823728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f187-a507-4559-a33a-350f7f16c292 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.825061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 142/298] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:08.831422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d028-8425-4618-8c56-f4f95661f658 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.831701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d028-8425-4618-8c56-f4f95661f658 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.832096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d028-8425-4618-8c56-f4f95661f658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.846043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:08.865907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a664d028-8425-4618-8c56-f4f95661f658 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:08.867923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a664d028-8425-4618-8c56-f4f95661f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:08.876149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 157/299] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:08.939467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 143/300] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:08.989353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccb82-da26-41a4-a709-9293d0c3aa3a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.989956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccb82-da26-41a4-a709-9293d0c3aa3a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.990426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccb82-da26-41a4-a709-9293d0c3aa3a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:08.991993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 158/301] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:08.997171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:08.997454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:08.997733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.009648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ccb82-da26-41a4-a709-9293d0c3aa3a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.015797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:09.016970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:09.103535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 144/302] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:09.120800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ccb82-da26-41a4-a709-9293d0c3aa3a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.155351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.156019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.156298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.156566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.167127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:09.172041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:09.172891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:09.173672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 159/303] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:09.176560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.180318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.192220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.239032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.281823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1eba-2588-420f-b5a3-6fe7b8dc0bd5 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.288350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2543f-b91a-4215-8598-ee08f1c54234 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.316434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.317071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.317346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.317615 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.323378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.324045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.324324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.324601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.325290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] RBAC: Adding query filter params (name=reader) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:09.325595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.327026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.329081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:09.333936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:09.335077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:09.335957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 145/304] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:09.336628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.339856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 160/305] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:09.341879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.410125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.438868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb45ad9-bcb3-4a5e-8fbb-4e6d83977ab8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.441213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28cea1-959f-49c7-8c10-dbf8450f176f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.462174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.473185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.473813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.474079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.474202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.474346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.474676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.474984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.475250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.475450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.475897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:09.476192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.476911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.477529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.478460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 146/306] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.479241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.486886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 161/307] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.487551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.576672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c88ac-f28d-46bc-965a-695c52743590 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.582356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8eec1-4ea6-490c-94d4-989367fc5850 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.609477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.609913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.610107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.610347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.610617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] PATH_INFO: `/v3/users/5d45531b5aed43ef9ea82eea89d36d6e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.615197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.615817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.616089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.616355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.624640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] RBAC: Authorizing `identity:get_user(user_id=5d45531b5aed43ef9ea82eea89d36d6e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.626107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.626209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:09.627737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 147/308] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/users/5d45531b5aed43ef9ea82eea89d36d6e => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.628467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:09.631421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:09.632270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 162/309] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:09.633385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.682362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.729830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dd51e-ecee-44e0-b435-e87ef4b914e3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.730006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38794-3718-4c39-b2d7-eb4bca71afc0 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.762148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.762301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.762772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.762908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.763079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.763178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.763351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] PATH_INFO: `/v3/users/15f30caf99c84773b9735236f0e812c0` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.763457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] PATH_INFO: `/v3/groups` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.764031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.764488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] RBAC: Authorizing `identity:get_user(user_id=15f30caf99c84773b9735236f0e812c0)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.765470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.765911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.767528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 163/310] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/users/15f30caf99c84773b9735236f0e812c0 => generated 297 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.768299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.795499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 148/311] 10.222.0.25 () {60 vars in 1134 bytes} [Sat Sep 3 03:36:09 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:09.795499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.846001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.896261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf90829-5f79-4e69-9336-441b05e05efe None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.900360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ac475-95be-4605-bdd7-735b10a495b3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:09.911577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:09.936206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.936883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.937165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.937440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.938108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:09.938418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.939837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.941019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:09.941706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:09.942238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:09.942238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:09.943004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, user.id=5d45531b5aed43ef9ea82eea89d36d6e, scope.system=all) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:09.943316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:09.944742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:09.948714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 149/312] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.949520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:09.951132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 164/313] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/role_assignments?user.id=5d45531b5aed43ef9ea82eea89d36d6e&scope.system=all&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:09.955018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:10.076174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee5220-f48c-456f-a2de-73181ca49108 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:10.086391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d287d-117f-41b0-ac8c-d833b7860f8e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:10.111833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:10.129760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:10.130425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:10.130710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:10.131003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] PATH_INFO: `/v3/users/48f8d350f36349f6bb990b27d2895f94` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:10.132667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] RBAC: Authorizing `identity:get_user(user_id=48f8d350f36349f6bb990b27d2895f94)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:10.134118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:10.136029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 165/314] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/users/48f8d350f36349f6bb990b27d2895f94 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:10.142284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:10.144062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:10.144726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:10.145000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:10.145265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:10.193341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:10.232886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:10.234717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:10.270898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc553-5381-4596-8678-eee5f9f89de2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:10.309284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:10.310568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:10.310906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:10.311182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] PATH_INFO: `/v3/system/users/48f8d350f36349f6bb990b27d2895f94/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:10.312581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=48f8d350f36349f6bb990b27d2895f94, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:10.313962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:10.334835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 166/315] 10.222.0.25 () {58 vars in 1334 bytes} [Sat Sep 3 03:36:10 2022] PUT /identity/v3/system/users/48f8d350f36349f6bb990b27d2895f94/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:10.377673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 150/316] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:10.384021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:10.453387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:10.502025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594dba3-8414-47a4-8c77-b851522fa5c4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:10.541149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:10.542167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:10.542641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:10.543002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:10.544265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:10.545765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d993b-8f98-4d9a-9311-64fa3f279ff3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:10.547735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 167/317] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:10 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:10.553375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:10.625061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:10.655697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c325a-234e-450a-aeca-573f02b0a9ae None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:10.690124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:10.690731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:10.691057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:10.691390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:10.692160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c, user.id=15f30caf99c84773b9735236f0e812c0) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:10.692467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:10.693915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee8a2c-611d-4659-b00a-51e8f0ad4bc4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:10.700342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 151/318] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:10 2022] GET /identity/v3/role_assignments?user.id=15f30caf99c84773b9735236f0e812c0&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:11.737822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dc80a-97fa-4e4d-be2e-c0c47ededa9b None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:11.738513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dc80a-97fa-4e4d-be2e-c0c47ededa9b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:11.738513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dc80a-97fa-4e4d-be2e-c0c47ededa9b None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:11.739955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 168/319] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:11.745032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01787d1-7710-4813-95df-773d790d5052 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:11.745436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01787d1-7710-4813-95df-773d790d5052 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:11.745550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01787d1-7710-4813-95df-773d790d5052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:11.765866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e01787d1-7710-4813-95df-773d790d5052 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:11.767081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e01787d1-7710-4813-95df-773d790d5052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:11.836341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 152/320] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:11.873814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd811b-9263-4fac-bdf0-dfe824f9253f None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:11.874192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd811b-9263-4fac-bdf0-dfe824f9253f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:11.874363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd811b-9263-4fac-bdf0-dfe824f9253f None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:11.875886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 169/321] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:11.880962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:11.881213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:11.881516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:11.899974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:11.901148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:11.961807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:11.970564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 153/322] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:12.100954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd811b-9263-4fac-bdf0-dfe824f9253f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.207082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd811b-9263-4fac-bdf0-dfe824f9253f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.230276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:12.237180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.246543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.247308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.247582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.247848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.249040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:12.250466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:12.258847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 170/323] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:12.262513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.332924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4698c-a8b6-4e7c-a602-15de48836aa1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.359833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde8109-0d28-44e0-b442-b7489523df8b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.365010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.365639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.365914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.366175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.376479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-d393855d-0885-4e53-bc5d-d5e7e9b5c50d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:12.381295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:12.382179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:12.382978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 154/324] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:12.384370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a7f7-6a09-4b94-a6ed-1072a86b8061 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.384370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a7f7-6a09-4b94-a6ed-1072a86b8061 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.384627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a7f7-6a09-4b94-a6ed-1072a86b8061 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.386014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 155/325] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:12.388091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a7f7-6a09-4b94-a6ed-1072a86b8061 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.395848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.396465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.396741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.397004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] PATH_INFO: `/v3/groups` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.397634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:12.399067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4c8e-050f-44b0-91c1-5e5ba7790c39 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:12.417700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 171/326] 10.222.0.25 () {60 vars in 1134 bytes} [Sat Sep 3 03:36:12 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:12.418892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.419176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.419445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.448437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:12.449670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:12.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:12.506529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:12.509008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a7f7-6a09-4b94-a6ed-1072a86b8061 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.531161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 172/327] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:12.553658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.554318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.554594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.554897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.555601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:12.555899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:12.557315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:12.570838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 156/328] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:12.572687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.680005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf1bc-2557-48af-a1e6-0a23a2e0d3e8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.717747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.718386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.718662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.718958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] PATH_INFO: `/v3/users/5d45531b5aed43ef9ea82eea89d36d6e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.720097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] RBAC: Authorizing `identity:get_user(user_id=5d45531b5aed43ef9ea82eea89d36d6e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:12.721537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:12.723630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 173/329] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/users/5d45531b5aed43ef9ea82eea89d36d6e => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:12.729204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.799285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:12.840287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac6ba1-a62e-4167-931e-3d4dfa12af38 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.860987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:12.866757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:12.888135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.888135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.888396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.888636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] PATH_INFO: `/v3/system/users/5d45531b5aed43ef9ea82eea89d36d6e/roles/7612f61be5c745b6927ee4bea9f4eeba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.890705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d45531b5aed43ef9ea82eea89d36d6e, role_id=7612f61be5c745b6927ee4bea9f4eeba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:12.892153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:12.910072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 157/330] 10.222.0.25 () {58 vars in 1334 bytes} [Sat Sep 3 03:36:12 2022] PUT /identity/v3/system/users/5d45531b5aed43ef9ea82eea89d36d6e/roles/7612f61be5c745b6927ee4bea9f4eeba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:12.932107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf90b-02a4-436b-a80a-829e89f3d18f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:12.972677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:12.973322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:12.973596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:12.973861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:12.986957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:12.992743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:12.993673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:12.994466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:12.998966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 174/331] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:13.000546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:13.112631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb09a9-a72c-4a4b-8223-ab2d39c128bf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:13.152020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:13.152635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:13.152913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:13.153184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:13.153850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:13.154149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:13.155648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:13.164442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 158/332] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:13.170345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:13.295515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b543198-2c25-43f7-90d8-e1adbaba6891 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:13.306473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:13.330490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:13.331162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:13.331502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:13.333550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] PATH_INFO: `/v3/users/48f8d350f36349f6bb990b27d2895f94` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:13.334724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] RBAC: Authorizing `identity:get_user(user_id=48f8d350f36349f6bb990b27d2895f94)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:13.336368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdce5-40d8-498f-ad12-e19a3c5cc521 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:13.337953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 175/333] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:13 2022] GET /identity/v3/users/48f8d350f36349f6bb990b27d2895f94 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:13.344168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:13.415352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:13.480067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b7b7b-5290-4ad2-8f9b-8665d7376a71 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:13.525169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:13.525705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:13.526042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:13.526313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:13.527122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] RBAC: Adding query filter params (role.id=87b7a008126642369351bd284f3e8354, user.id=48f8d350f36349f6bb990b27d2895f94, scope.system=all) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:13.527433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:13.528950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbb831-d621-42f0-9552-13f6f8bcc756 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:13.541371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 159/334] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Sep 3 03:36:13 2022] GET /identity/v3/role_assignments?user.id=48f8d350f36349f6bb990b27d2895f94&scope.system=all&role.id=87b7a008126642369351bd284f3e8354 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:14.867191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036390cc-da62-4d20-a950-335f6754490e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:14.867710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036390cc-da62-4d20-a950-335f6754490e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:14.867710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036390cc-da62-4d20-a950-335f6754490e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:14.869317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 176/335] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:14.880987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a09a-4b1e-4443-910d-6aaa046bb849 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:14.881492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a09a-4b1e-4443-910d-6aaa046bb849 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:14.881570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a09a-4b1e-4443-910d-6aaa046bb849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:14.904699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4345a09a-4b1e-4443-910d-6aaa046bb849 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:14.905888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4345a09a-4b1e-4443-910d-6aaa046bb849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:14.983792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 160/336] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:15.374464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286b930-8a61-437f-b372-648818ff3e6f None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.374874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286b930-8a61-437f-b372-648818ff3e6f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.375051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286b930-8a61-437f-b372-648818ff3e6f None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.376622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 177/337] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:15.382441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6008-70f5-49a3-811e-a4c29f38e91d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.382728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6008-70f5-49a3-811e-a4c29f38e91d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.383027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6008-70f5-49a3-811e-a4c29f38e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.405937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-516b6008-70f5-49a3-811e-a4c29f38e91d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:15.407138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-516b6008-70f5-49a3-811e-a4c29f38e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:15.436840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db548c2-52ea-4ac2-8355-d37827e75ea3 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.437144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db548c2-52ea-4ac2-8355-d37827e75ea3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.437393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db548c2-52ea-4ac2-8355-d37827e75ea3 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.437875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:15.439126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 178/338] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:15.466946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc5e2-32a5-4527-b120-915b4af0a6f5 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.467279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc5e2-32a5-4527-b120-915b4af0a6f5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.467505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc5e2-32a5-4527-b120-915b4af0a6f5 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.469017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 179/339] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:15.469883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.470161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.470433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.489259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 161/340] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:15.491487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.496319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.496619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.518486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:15.519688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:15.538044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:15.539341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:15.642926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 180/341] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:15.643885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:15.684410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 162/342] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:15.773909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb0c11-287a-4447-bbce-b02edf6b1dbd None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:15.783859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:15.813890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:15.814526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:15.814816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:15.815107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:15.825843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:15.830585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:15.831548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:15.832632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 181/343] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:15.840285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:15.871457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:15.983516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:15.994131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1aafd-5aed-4ad5-bcb3-3e3d51df9b07 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.032957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.033491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.033764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.034034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.034970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:16.035274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.036648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.042512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8dd3-23ae-44b2-adca-7551fa16e9b0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.045678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 163/344] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.051741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.084275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.084912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.085197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.085457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.097230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:16.102115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:16.102991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 03:36:16.103788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:16.104424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 182/345] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:16.119948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.156131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.168622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3430d3-b68c-46cd-9e15-7acaf074fe7a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.210672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.211548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.211958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.212367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.242371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518255-3749-4ca6-8b04-6dc5d8221522 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.245370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.280887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.281612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.281952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.282287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.294981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:16.299010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:16.300391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:16.301478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:16.302355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:16.303526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.304156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 183/346] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:16.304995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.411473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 164/347] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:36:16.412551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.417729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b31f9f-4c5e-4883-87ef-c6c284794f19 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.417729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.454555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.455864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.455864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.455864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.456402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:16.457556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.458287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.467872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 184/348] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.468790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.480130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.528650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e740582-5658-40a5-9f2a-a4c278691ff7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.576160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.577045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.577321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.577596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.585344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa9154-24dc-45c2-9b8e-158b35fa2ee8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.588922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:16.593992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:16.594980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:16.596174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 165/349] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:16.597028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.629538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.630984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.631160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.631424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.632112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:16.633006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.633793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.645497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 185/350] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.645497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.663545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.729610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719752d2-92eb-45ac-8c03-1d37d616e06c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.747727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.771848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.772496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.772773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.773044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.774190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.775712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa27b56-af01-410c-a567-8aafb3a01df5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.776351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a1839-9c7a-4cca-9a85-fe2b6fec387b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.777570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 166/351] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.778490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b87a15-715c-44b2-9f8a-9ff6b994e68a None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.778772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b87a15-715c-44b2-9f8a-9ff6b994e68a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.780866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b87a15-715c-44b2-9f8a-9ff6b994e68a None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.782272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 167/352] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:16.783905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b87a15-715c-44b2-9f8a-9ff6b994e68a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.823231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.823960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.824239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.824729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] PATH_INFO: `/v3/users/5d45531b5aed43ef9ea82eea89d36d6e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.825656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] RBAC: Authorizing `identity:get_user(user_id=5d45531b5aed43ef9ea82eea89d36d6e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.827406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.828989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 186/353] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/users/5d45531b5aed43ef9ea82eea89d36d6e => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.829601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:16.932171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:16.937814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b87a15-715c-44b2-9f8a-9ff6b994e68a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.958210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe2cbf-b56d-40ed-a158-584cd06d0698 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:16.975261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:16.975874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:16.976202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:16.976467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:16.977177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:16.977474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:16.978891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:16.988332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 168/354] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:16.989165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.005487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.006210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.006489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.006768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.044597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.090323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.092043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.105264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b31ee-a9b9-4012-8bbb-d006d0701b34 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.146133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.147331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.147600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.147864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.158255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.159880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a448ba-58ae-4334-adec-84d89171dee0 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.161907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 169/355] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:17.162929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.163211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.163576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.182219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:17.183475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:17.185756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.210944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 187/356] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:17.211867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.278937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 170/357] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:17.278937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.328993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.343438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efea2-ce8c-4591-8032-7b7c0e93ef72 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.386833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.387766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.388213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.388652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.389542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] RBAC: Adding query filter params (role.id=7612f61be5c745b6927ee4bea9f4eeba, user.id=5d45531b5aed43ef9ea82eea89d36d6e, scope.system=all) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:17.390028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.391665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.394728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c2d46-8e61-47a3-bba3-5754f56e429e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.406464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 188/358] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/role_assignments?user.id=5d45531b5aed43ef9ea82eea89d36d6e&scope.system=all&role.id=7612f61be5c745b6927ee4bea9f4eeba => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:17.407614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.454059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.455081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.455551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.456059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.463286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.546615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365b50-3832-411f-b524-e2b405984908 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.558549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.561153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed947-81de-41f7-93f2-4749938d3fe1 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.593833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.594396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.594675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.594978 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.595710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=87b7a008126642369351bd284f3e8354, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:17.596014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.597799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.605117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 189/359] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=87b7a008126642369351bd284f3e8354 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:17.606036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.617268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.707420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 171/360] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:17.709067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992c2f-ef53-431b-83c9-eb8867e5d78e None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.709648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992c2f-ef53-431b-83c9-eb8867e5d78e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.709924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992c2f-ef53-431b-83c9-eb8867e5d78e None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.711302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 172/361] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:17.712109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992c2f-ef53-431b-83c9-eb8867e5d78e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.766255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8655037-d185-4d70-805e-8055f9298afd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.768127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.819370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.821016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.821491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.821951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.823998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:17.825929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:17.828583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 190/362] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:17.829542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:17.849755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992c2f-ef53-431b-83c9-eb8867e5d78e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:17.891943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:17.892755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.893121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.893389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.901261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:17.909089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-691942ef-f010-46dc-9791-ff286fceab43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:17.912796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:17.913698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:17.914509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 173/363] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:17.915636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:17.915810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:17.916085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:17.942114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:17.943428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:17.965936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d957e1-4796-4db8-96a7-00f92bb810bb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.011553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.012193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.012470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.012739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.014265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.015757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.017927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 191/364] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:17 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.018130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.034077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 174/365] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:18.036348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.067092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.140950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebf864-a129-46a3-b78e-83c602509679 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.155643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca5d2c-b101-46f0-845b-bd6d0142929c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.178323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.178988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.179267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.179538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.180702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.182141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.185296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 192/366] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.186063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.186393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.200363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.201554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.201919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.202551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.203357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:18.203754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.205964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.218131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 175/367] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.218952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.308797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09e5e-8df2-46e5-9f58-477ba10d349d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.319435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.350628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109fc8-cc6f-4d8e-8689-0e92c6330a2a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.353015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.353724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.354072 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.354412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.356325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.358200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.359953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 193/368] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.360822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.394887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.395407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.395688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.395959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.396692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=87b7a008126642369351bd284f3e8354, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:18.396996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.398438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.404177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 176/369] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=87b7a008126642369351bd284f3e8354 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.405786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.477817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.499021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144f61-9bec-4c30-98f1-b49adf9ba42f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.539191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.540124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.540575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.541046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.544300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fd39-eb27-4d1d-b1cb-aaccd481199f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.590437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.591138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.591412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.591685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.592173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.613189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:18.618779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:18.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:18.620574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:18.621348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 177/370] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:18.622919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.638837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.640628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.730137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a352f-69ba-407f-8cf6-f39df0fa32b6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.744750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.746022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 194/371] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:18.751831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.766352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:18.767012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:18.767292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:18.767566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:18.768304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:18.768620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:18.770164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:18.779621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 178/372] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:18.779621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:18.923054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:18.949781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860addf-035a-41c1-ba24-1b7334980228 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:18.974221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84da0-84fb-4144-a902-80ebae9370e6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.018159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.019214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.020226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.021051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.021583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.022924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.023909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.024302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.024758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.026401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.027303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:19.027826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.029388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.031867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 179/373] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.045464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 195/374] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.052201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.061794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.119204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:19.177249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ad0a-ff18-451d-8a60-87a43a33ebf1 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.185490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752a1fe-d001-4f5d-b017-97f90aec2b04 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.220395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.221412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.221930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.222490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.223836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.225670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.231990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 196/375] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.234237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.238551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:19.252119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.252757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.253033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.253340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.344291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0bdf-50a1-4898-9315-30c4363c5f14 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.369022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.370490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.387646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059f095-f849-4daa-964c-8228417d033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.390503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059f095-f849-4daa-964c-8228417d033a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.391011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059f095-f849-4daa-964c-8228417d033a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.391483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059f095-f849-4daa-964c-8228417d033a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.392430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059f095-f849-4daa-964c-8228417d033a None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:19.394981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059f095-f849-4daa-964c-8228417d033a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.396697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059f095-f849-4daa-964c-8228417d033a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.415331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 197/376] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.467206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:19.530976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 180/377] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:19.536925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059f095-f849-4daa-964c-8228417d033a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.619243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:19.660259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059f095-f849-4daa-964c-8228417d033a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.702477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.703190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.703472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.703746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] PATH_INFO: `/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.704990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] RBAC: Authorizing `identity:get_project(project_id=1fa3555aedbc4a4eaa2e41bd6d2eeab3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.706508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8ca3e-3a97-4d86-83a3-52d50ab4f6ab None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.709510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 198/378] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.714312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.794787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7574-0334-4b98-a434-3bdd9bafc8f7 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.795147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7574-0334-4b98-a434-3bdd9bafc8f7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.795381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd7574-0334-4b98-a434-3bdd9bafc8f7 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.796857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 199/379] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:19.803742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.804317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.804566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.828140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:19.829498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:19.848023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca0ade-417e-48f5-9a4f-c7881736c336 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:19.914659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:19.915241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:19.915517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:19.915781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] PATH_INFO: `/v3/groups/5c4604dbf8b94239a9edbf2587f407db` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:19.924814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4604dbf8b94239a9edbf2587f407db)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:19.926743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:19.928400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 181/380] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/groups/5c4604dbf8b94239a9edbf2587f407db => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:19.935072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:19.944853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 200/381] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:19.954432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:20.052464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45adea-083c-4e67-ac16-55a5352c9f74 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:20.086827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:20.087447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.087713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.087980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.088704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] RBAC: Adding query filter params (group.id=5c4604dbf8b94239a9edbf2587f407db, role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=1fa3555aedbc4a4eaa2e41bd6d2eeab3) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:20.089067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:20.090587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:20.096744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 182/382] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:19 2022] GET /identity/v3/role_assignments?group.id=5c4604dbf8b94239a9edbf2587f407db&scope.project.id=1fa3555aedbc4a4eaa2e41bd6d2eeab3&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:20.387760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:20.470211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:20.512111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4a19-8513-469b-bc05-1cb141f40cf9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:20.549418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:20.550142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.550419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.550686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.561078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:20.565976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:20.566903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:20.567688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 201/383] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:20.575933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:20.662741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:20.693609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d6102-5955-43b0-9d11-be3d746f1cb9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:20.733614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:20.734791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.735255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.735717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.736547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:20.737012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:20.738775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb212c2-70f7-425b-8472-e3190519c764 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:20.751441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 183/384] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:20.757185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:20.759512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d6cc-547d-4aa5-95c3-9c18b74c1596 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.759790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d6cc-547d-4aa5-95c3-9c18b74c1596 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.760061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d6cc-547d-4aa5-95c3-9c18b74c1596 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.761719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 184/385] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:20.767301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.767518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.767778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.788736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:20.790076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:20.930380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 185/386] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:20.943084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b6c4b-171d-4125-8817-b859e51a5317 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:20.978986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:20.979635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:20.979912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:20.980180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:20.981286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:20.982034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:20.985672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:20.987859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 202/387] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:20 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:20.993686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.075747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:21.114657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531b01d-1431-4f8c-8d94-b468f292c89c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:21.155880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:21.156516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.156838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.157108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.158203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:21.159760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:21.161732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 186/388] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:20 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:21.167450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.280053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b561-a4f0-4b96-9f4f-5fe9a285cd67 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:21.314497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:21.315198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.315468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.315802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/groups/887ca353e8e9453cb4b82c6701e1434e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.317674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, group_id=887ca353e8e9453cb4b82c6701e1434e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:21.319145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20925f01-d3af-4c2e-a152-f378b1e14f17 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:21.333140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 203/389] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Sep 3 03:36:21 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/groups/887ca353e8e9453cb4b82c6701e1434e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:21.411877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.528856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b566-4666-4fc2-8541-50165ebb3b81 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:21.574489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:21.575169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.575440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.575703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.577782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:21.589028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:21.593471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66713c-2906-49cb-9a5e-fd02b842ef2f None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.593925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66713c-2906-49cb-9a5e-fd02b842ef2f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.594115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66713c-2906-49cb-9a5e-fd02b842ef2f None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.595660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 204/390] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7913e68d-d887-457d-8441-f71bbb5cab7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:21.598071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:21.599020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:21.600441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 187/391] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:21.608562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66713c-2906-49cb-9a5e-fd02b842ef2f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.608931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.609213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.609481 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.630316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:21.631562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:21.699324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:21.726930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 188/392] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:21.749344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66713c-2906-49cb-9a5e-fd02b842ef2f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:21.784423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:21.785031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.785305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.785700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.786360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:21.786701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:21.788132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:21.797036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 205/393] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:21.802315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.905799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb503-483a-4e55-83ac-6f499084acc6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:21.935813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:21.949597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:21.950301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.950572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.950849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.951966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:21.953407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40dc2ef-c2f8-4113-a7b0-e91132e7bdc9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:21.955204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 189/394] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:21 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:21.960588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:21.977523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427af47-e3f4-4fb5-ab8f-af02d5ec2726 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.977818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427af47-e3f4-4fb5-ab8f-af02d5ec2726 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.978069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427af47-e3f4-4fb5-ab8f-af02d5ec2726 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:21.979907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 190/395] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:21.986967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:21.987248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:21.987560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.019333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:22.020773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:22.088134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced05bfc-f941-4a35-9d24-1ffd5c9ab633 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.107088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 191/396] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:22.107968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.108985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.132503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.133793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.134712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.135254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.136993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.139489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.144898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 206/397] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:21 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:22.148636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.272435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90c00-3fa0-4337-87d1-c787ee4cf40b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.307869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073da157-f694-4b97-8198-2dd106c881d4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.311838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.312700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.313168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.314182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/groups/887ca353e8e9453cb4b82c6701e1434e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.322825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, group_id=887ca353e8e9453cb4b82c6701e1434e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.322825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.346776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.359938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 207/398] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Sep 3 03:36:22 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/groups/887ca353e8e9453cb4b82c6701e1434e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:22.370613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.371562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.371976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.372534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.389056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:22.394515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:22.395740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:22.396772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:22.404369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 192/399] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:22.410486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.439499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.481220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.555836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73759e46-76e5-4f91-96f0-420a517f52bc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.586616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08bfc-4372-4e5a-8ec6-d813a2270527 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.599384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.600099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.600277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.600555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.601511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:22.601822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.604203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.615172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 208/400] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:22.621669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.638201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.640831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.640831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.640831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.654137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-afda66b9-5980-4c6b-b96b-ffc825576766 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:22.659259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:22.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:22.661007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 193/401] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:22.662149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64b806-2a4b-4a78-982b-2125aa7ac389 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.662344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64b806-2a4b-4a78-982b-2125aa7ac389 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.662620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64b806-2a4b-4a78-982b-2125aa7ac389 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.664699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 194/402] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:22.669476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64b806-2a4b-4a78-982b-2125aa7ac389 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.713991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.734794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277259-1009-479e-87d0-8737f5d14c97 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.743187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.772273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64b806-2a4b-4a78-982b-2125aa7ac389 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.773845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.774777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.775280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.775733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.777039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.778663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df0f8-9edf-4f82-922c-5d9a8aa095c3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.781045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 209/403] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:22.782724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.783364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.783876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.804123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:22.805493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:22.807519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.808284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.808712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.809108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.809901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:22.810417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.812010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.821293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 195/404] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:22.822357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.911055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 210/405] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:22.912232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:22.946535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c58ea2-3ecb-4d79-826c-a3323da98173 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:22.949882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:22.991716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:22.992365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:22.992698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:22.993113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:22.994748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:22.996385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:22.998520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 196/406] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:23.005481 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.085562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c086f3-167c-443c-943e-df7586f8d358 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.089472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:23.163207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6012-b4f2-4414-ba0e-770f3f46c53f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.166434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.167324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.167825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.168317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.171529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.179648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.181741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 211/407] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:22 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:23.187936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.218682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.219384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.219970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.220252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/groups/887ca353e8e9453cb4b82c6701e1434e/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.223140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] RBAC: Authorizing `identity:create_grant(project_id=805932c9d70c4b1b95a43f25dce5a4ab, group_id=887ca353e8e9453cb4b82c6701e1434e, role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.224932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.247123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 197/408] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Sep 3 03:36:23 2022] PUT /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab/groups/887ca353e8e9453cb4b82c6701e1434e/roles/c566cf204ec94aaba0a8466d98161f8d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:36:23.248471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.319783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:23.360922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f76d6-d3ca-43f1-a3d6-4960d89552fc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.427511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.428160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.428435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.428733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.431014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.432480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.434267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 212/409] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:23 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:23.441829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.446908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5296bc-c2b6-4845-976b-e5189e124ecb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.492011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.492710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.492989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.493310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.507275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:23.507422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:23.513492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:23.514404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:23.519781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 198/410] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:23.522345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.596990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96dff-ccbf-46b7-805e-d434ea526b04 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.669747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.670382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.670603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.670890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/groups/887ca353e8e9453cb4b82c6701e1434e/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.675844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] RBAC: Authorizing `identity:create_grant(project_id=a840ccbd6e3a4fa0b825e2e5031a081c, group_id=887ca353e8e9453cb4b82c6701e1434e, role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.677641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa6727-c9b2-4783-b525-0f602b911e59 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.689130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:23.696792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdaa7a-8d92-4877-830e-aceb9226fb03 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.700174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 213/411] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Sep 3 03:36:23 2022] PUT /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c/groups/887ca353e8e9453cb4b82c6701e1434e/roles/c566cf204ec94aaba0a8466d98161f8d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:23.719262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30987e0e-d03b-44e2-98c1-c4e9c192a8ca None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.719572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30987e0e-d03b-44e2-98c1-c4e9c192a8ca None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.719814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30987e0e-d03b-44e2-98c1-c4e9c192a8ca None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.721263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 214/412] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:23.726647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147021d3-c109-4014-9f14-3df86feef970 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.727092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147021d3-c109-4014-9f14-3df86feef970 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.727464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147021d3-c109-4014-9f14-3df86feef970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.747155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-147021d3-c109-4014-9f14-3df86feef970 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:23.748503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-147021d3-c109-4014-9f14-3df86feef970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:23.758347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.759109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.759401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.759673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.760443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:23.760755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.762213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.782227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 199/413] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:23.788423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:23.822383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 215/414] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:23.942124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5b04-3c59-4af4-bcf0-728be09f571e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:23.979399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:23.980044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:23.980319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:23.980593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] PATH_INFO: `/v3/groups/5c4604dbf8b94239a9edbf2587f407db` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:23.982001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4604dbf8b94239a9edbf2587f407db)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:23.983501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:23.985769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 200/415] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:23 2022] GET /identity/v3/groups/5c4604dbf8b94239a9edbf2587f407db => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:23.993029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147021d3-c109-4014-9f14-3df86feef970 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:24.026103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:24.115537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147021d3-c109-4014-9f14-3df86feef970 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:24.125573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:24.159630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:24.176260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:24.176923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.177258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.177540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] PATH_INFO: `/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.183310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] RBAC: Authorizing `identity:get_project(project_id=1fa3555aedbc4a4eaa2e41bd6d2eeab3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:24.184822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:24.186655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 216/416] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:23 2022] GET /identity/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:24.192732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:24.258951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5f23f-338a-4196-9ef2-eca6b2d21868 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:24.327533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:24.328174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.328454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.328734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.347225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:24.352796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:24.353689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:24.354536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:24.357336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 201/417] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:24.360444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:24.382706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:24.387298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be557c-bdeb-4442-876d-52fcf7a4f01b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:24.456771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:24.457353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.457637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.457910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] PATH_INFO: `/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3/groups/5c4604dbf8b94239a9edbf2587f407db/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.463423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa3555aedbc4a4eaa2e41bd6d2eeab3, group_id=5c4604dbf8b94239a9edbf2587f407db, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:24.464876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:24.497654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 217/418] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Sep 3 03:36:24 2022] PUT /identity/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3/groups/5c4604dbf8b94239a9edbf2587f407db/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:24.517398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:24.537164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2add11-29fd-479a-a2ca-8ae45c23425d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:24.572206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:24.572824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.573104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.573379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.574064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:24.574367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:24.575800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1da7e-8bb1-4f14-b0c5-8cf43196b5e2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:24.589277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 202/419] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:24.595178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:24.681291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b362d3-035f-4227-b76c-18099f3c77db None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.681631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b362d3-035f-4227-b76c-18099f3c77db None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.681830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b362d3-035f-4227-b76c-18099f3c77db None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.683239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 203/420] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:24.698156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.698463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.698704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.720514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:24.726841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:24.749259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a5ca3-268c-440b-91f4-4faf9618717b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:24.796817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:24.797560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:24.797935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:24.798211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:24.803508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 204/421] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:24.827065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:24.872682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:24.874200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:24.983248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 218/422] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:24.985254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.076509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:25.108053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c1b20-396c-4763-9551-7814b1bf5c73 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.154134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.154832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.155116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.155387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.156673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:25.158243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:25.160222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 205/423] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:24 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:25.165690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.229964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:25.255097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.308951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbb5d-6c5d-4321-a772-f93ee1609ccb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.359614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.360364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.360638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.360902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.362462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:25.363924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:25.365782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 219/424] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:25 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:25.371415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.464877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c5ee-270d-4e77-a488-ff25c1ce5e5a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.478143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a0491-7d5b-45f2-b6b3-e107cb2a640a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.505514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:25.515879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.516484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.516800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.517074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.517797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=87b7a008126642369351bd284f3e8354, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:25.518100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:25.519624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:25.526138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 220/425] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:25 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=87b7a008126642369351bd284f3e8354 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:25.553137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.553798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.554074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.554342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.575886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:25.580880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:25.583392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:25.585094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 206/426] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:25.587937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.720184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f791a7-d663-46ae-b425-3e92767494f8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.755080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.755900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.756071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.756229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.756841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] RBAC: Adding query filter params (name=member) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:25.757137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:25.758494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bc08d-af89-42e9-b9cf-84a72463b616 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:25.767232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 221/427] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:36:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:25.773425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:25.777815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1410f45-0d5f-4fa2-b4ff-fda07f180e77 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.777815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1410f45-0d5f-4fa2-b4ff-fda07f180e77 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.777815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1410f45-0d5f-4fa2-b4ff-fda07f180e77 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.778388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:25.778503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 222/428] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:25.787204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838dd97-3da1-4576-b8bd-45cced10f984 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.787204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838dd97-3da1-4576-b8bd-45cced10f984 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.787204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838dd97-3da1-4576-b8bd-45cced10f984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.805674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4838dd97-3da1-4576-b8bd-45cced10f984 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:25.807641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4838dd97-3da1-4576-b8bd-45cced10f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:25.878343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:25.893726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da67b1-025f-46c1-a27e-31b4dab6bd2c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:25.918830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 223/429] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:25.938057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:25.938659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.938967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.939236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.987551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e374-99f8-4701-be9c-04ab2b88aa35 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:25.987856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e374-99f8-4701-be9c-04ab2b88aa35 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:25.988107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e374-99f8-4701-be9c-04ab2b88aa35 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:25.989678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 224/430] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:25.999879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.000204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.000430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.010883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.012395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.016343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.027868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:26.029085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:26.111538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 225/431] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:26.138068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 207/432] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:36:26.143518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.222760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.267396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cef777-5cd4-4911-8df7-fabd165efc81 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.285048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.309485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.310391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.310826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.311277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.312568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.314204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.316427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 226/433] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:26.322635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.344853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416f32f-ca5c-4f2d-abbe-3ad02f56905d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.348468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.390387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.391469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.391749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.403379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:26.408105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:26.408985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:26.409844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 208/434] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:26.410515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.462001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c2b5f-a5ec-4ba5-8bf7-b7190c6af787 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.506083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.506725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.507045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.507315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.510046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.511548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.513263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 227/435] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:26.515565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.535161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f2585-88dc-4041-9462-61d8a8f11ba2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.566719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.568930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.569583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.569901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.570104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.581420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:26.586273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:26.587219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:26.588018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 209/436] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:26.588660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.614582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.626998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78ba0a-f65f-47dc-8b84-09bc96397314 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.661785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.662390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.662662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.662961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.663614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:26.663914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.665305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.678883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 228/437] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:26.678883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.698098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265cc6-7fbb-4284-873c-c48d96f2b1cf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.731884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.732480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.732756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.733030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.733750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=87b7a008126642369351bd284f3e8354, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:26.734053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.735518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.741433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 210/438] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=87b7a008126642369351bd284f3e8354 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:26.747874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.793918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32002a-5cd0-4e69-a086-7cb9e3fff899 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.806887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.835321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.835969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.836244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.836513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.837241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:26.837542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:26.839009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:26.852853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 229/439] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:26.853889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:26.888693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:26.905187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2941e64-6a49-47e6-a0f5-2622241db47c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.943291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.943906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.944178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.944442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:26.965369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f431f8-4a9c-48df-999a-710608e9534e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:26.997575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:26.998173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:26.998448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:26.998714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.011442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.012931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c29c4-d6a7-449c-a91c-daa87391c74b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.037862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.064512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.065939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796267d4-46e9-4e18-a2f2-769ea9dc5296 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.096267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.112321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 211/440] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:27.113737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4497a1-5050-4d18-a29d-5e23cd0e5e49 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.114021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4497a1-5050-4d18-a29d-5e23cd0e5e49 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.114294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4497a1-5050-4d18-a29d-5e23cd0e5e49 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.119956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 212/441] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:27.120726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4497a1-5050-4d18-a29d-5e23cd0e5e49 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.163943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 230/442] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:27.165187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.165467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.165736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.183355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:27.184526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:27.227720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4497a1-5050-4d18-a29d-5e23cd0e5e49 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.260159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 231/443] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:27.261018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.263202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.267846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.268601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.268881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.269152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] PATH_INFO: `/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.270336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] RBAC: Authorizing `identity:get_project(project_id=805932c9d70c4b1b95a43f25dce5a4ab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.271827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.273446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 213/444] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/projects/805932c9d70c4b1b95a43f25dce5a4ab => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:27.278697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.340367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.376590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a065e9-f36d-4be6-acbf-30d6756028e6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.390607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2864c-6313-47fa-81dd-bd177233a873 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.411523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.412198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.412476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.412741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] PATH_INFO: `/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.413832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] RBAC: Authorizing `identity:get_project(project_id=a840ccbd6e3a4fa0b825e2e5031a081c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.415301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.417090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 232/445] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/projects/a840ccbd6e3a4fa0b825e2e5031a081c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:27.422254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.425902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.426537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.426826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.427108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.428205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.429638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.431350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 214/446] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:27.436676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.492969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.523599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3133-d882-4767-91a6-129da473c42e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.543413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e55cf-474d-4593-8132-16acd5881527 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.556773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.557402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.557676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.557946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] PATH_INFO: `/v3/groups/887ca353e8e9453cb4b82c6701e1434e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.559077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] RBAC: Authorizing `identity:get_group(group_id=887ca353e8e9453cb4b82c6701e1434e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.560508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.562065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 233/447] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/groups/887ca353e8e9453cb4b82c6701e1434e => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:27.562823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.565520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.577529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.578146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.578398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.578668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.579427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:27.579730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.581262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.595573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 215/448] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=805932c9d70c4b1b95a43f25dce5a4ab&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:27.595573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.691703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b5359-2175-4b48-9cf3-ab79f53f3daf None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.701469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa4aa9-068b-47af-9866-1ae6c8ec8b2f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.728226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.728861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.729948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.729948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.738075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.738212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.738212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.738476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.738739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.739581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] RBAC: Adding query filter params (group.id=887ca353e8e9453cb4b82c6701e1434e, role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:27.739913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.741379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.743188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:27.747671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 216/449] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/role_assignments?group.id=887ca353e8e9453cb4b82c6701e1434e&scope.project.id=a840ccbd6e3a4fa0b825e2e5031a081c&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:27.749084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:27.750013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:27.750796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 234/450] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:27.756184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:27.817927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:27.875208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd80e-0168-4738-81c2-0c84b994e7dc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:27.913129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:27.913744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:27.914016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:27.914287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:27.914982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:27.915279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:27.916664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:27.926213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 217/451] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:27.931921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:28.039450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55824-5801-4053-abcf-bd5bc7a78359 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:28.074580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:28.075219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:28.075498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:28.075774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:28.139642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:28.143551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:28.145176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:28.242955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 235/452] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:28.248813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:28.351066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a5b5a-ea18-442a-9cc7-d6f522b5e5b6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:28.365013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:28.387776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:28.388393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:28.388669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:28.388942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] PATH_INFO: `/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:28.390238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa3555aedbc4a4eaa2e41bd6d2eeab3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:28.391654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:28.393251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 218/453] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:36:28 2022] GET /identity/v3/projects/1fa3555aedbc4a4eaa2e41bd6d2eeab3 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:28.399039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:28.494128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:28.497846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e89c2-23f3-4de4-be95-907db3c6bb22 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:28.532411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:28.533020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:28.533296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:28.533565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] PATH_INFO: `/v3/groups/5c4604dbf8b94239a9edbf2587f407db` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:28.534832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4604dbf8b94239a9edbf2587f407db)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:28.536265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b131-0cc8-4900-b2ce-df9d6b4a9a33 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:28.538462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 236/454] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Sep 3 03:36:28 2022] GET /identity/v3/groups/5c4604dbf8b94239a9edbf2587f407db => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:28.544938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:28.651392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206d4de-92bd-4f07-9662-67fcad55af69 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:28.687724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:28.688308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:28.688579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:28.688847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:28.689566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] RBAC: Adding query filter params (group.id=5c4604dbf8b94239a9edbf2587f407db, role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=1fa3555aedbc4a4eaa2e41bd6d2eeab3) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:28.689869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:28.692295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbea2e9-565b-4c0f-a9f7-8d4f38849622 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:28.702201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 219/455] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Sep 3 03:36:28 2022] GET /identity/v3/role_assignments?group.id=5c4604dbf8b94239a9edbf2587f407db&scope.project.id=1fa3555aedbc4a4eaa2e41bd6d2eeab3&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:30.918510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc389-93ff-4563-ab6a-5006267a08cc None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:30.919110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc389-93ff-4563-ab6a-5006267a08cc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:30.919395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc389-93ff-4563-ab6a-5006267a08cc None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:30.921082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 237/456] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:30.926416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db66bf-ec3d-49ba-b5b0-bad80d0ee542 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:30.926883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db66bf-ec3d-49ba-b5b0-bad80d0ee542 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:30.926982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db66bf-ec3d-49ba-b5b0-bad80d0ee542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:30.951820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-50db66bf-ec3d-49ba-b5b0-bad80d0ee542 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:30.953025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-50db66bf-ec3d-49ba-b5b0-bad80d0ee542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:31.006094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:31.032953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 220/457] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:31.118307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4ff5-ca58-45f7-b6d2-70d691d66ec0 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.118307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4ff5-ca58-45f7-b6d2-70d691d66ec0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.118307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4ff5-ca58-45f7-b6d2-70d691d66ec0 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.118307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 238/458] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:31.123702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc26d1-f1c1-4586-bac8-248f2a76a2c1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.123990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc26d1-f1c1-4586-bac8-248f2a76a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.124266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc26d1-f1c1-4586-bac8-248f2a76a2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.151813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e8dc26d1-f1c1-4586-bac8-248f2a76a2c1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:31.153069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e8dc26d1-f1c1-4586-bac8-248f2a76a2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:31.235078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 221/459] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:31.378058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e4ff5-ca58-45f7-b6d2-70d691d66ec0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:31.444887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d04-8183-4324-bc63-47a226deaa17 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.445206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d04-8183-4324-bc63-47a226deaa17 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.445441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d04-8183-4324-bc63-47a226deaa17 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.447406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 222/460] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:31.452137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.452413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.452679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.470898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:31.472115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:31.537390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e4ff5-ca58-45f7-b6d2-70d691d66ec0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:31.540578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 223/461] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:31.571354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:31.609198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:31.609837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.610115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.610383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.626954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:31.627237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:31.628804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:31.653276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:31.654371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:31.655348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:31.655348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:31.655348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:31.655348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 239/462] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:31.662204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:31.687173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:31.757428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b60e6-fe9a-40e4-800e-df922e1c14e3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:31.818569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff713bb-877f-4c83-a3dd-bea90ae1685b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:31.827314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:31.828091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.828370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.828703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.842871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:31.845327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:31.858014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:31.858625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:31.858972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:31.859416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:31.860194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:31.860653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:31.862236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:31.865954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:31.866957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:31.867812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:31.867812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:31.867812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:31.870488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 224/463] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:31.876016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:31.879695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 240/464] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:31.885118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:31.965467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:31.976028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:31.990899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700f5f-958c-4c46-b3ac-7f03a4cb68d2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:31.992265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266a7eb-6104-43a8-ac0c-1ea7f612193c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.026238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.026898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.027180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.027447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.028529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.029135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.029417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.029689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.030325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:32.030736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.032048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.039198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.040679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.044209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 241/465] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:32.045043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2c7d4d3c-633f-4740-b5c3-b5104bc0877c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:32.057422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:32.058541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:32.059590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:32.059590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:32.059590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:32.060955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 225/466] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:32.062281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f77a1-60e2-4135-9d2b-c26d0204f02b None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.062576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f77a1-60e2-4135-9d2b-c26d0204f02b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.062841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f77a1-60e2-4135-9d2b-c26d0204f02b None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.064229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 226/467] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:32.064754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f77a1-60e2-4135-9d2b-c26d0204f02b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:32.162839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410bdce-c4e2-4ac4-9905-bcf83b3ae7fe None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.172631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f77a1-60e2-4135-9d2b-c26d0204f02b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.199777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.200573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.200680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.200903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.201592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.203010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.208726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.209337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.209617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.209891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.210533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.212001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1079-c436-41c5-8a06-1d609999e049 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.213725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:32.222647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:32.228985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 242/468] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:32 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:32.229668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:32.244605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 227/469] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:32 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:32.246056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.246374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.246743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.267442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:32.268860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:32.398839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 228/470] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:32.434378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882a989-1763-4469-acc0-7ec454c559b4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.479429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.480331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.480618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.480892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.481852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:32.482158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.483630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc454-c987-49c3-b4ff-42247c881bce None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.497557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 243/471] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:32.503231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:32.615600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9e6b9-8338-4635-8e78-40de316be9d6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.641166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:32.653864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.654465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.654825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.655222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.656516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.656708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0cd83-e45a-4dae-b733-e1d5d70442ae None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.656987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0cd83-e45a-4dae-b733-e1d5d70442ae None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.657283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0cd83-e45a-4dae-b733-e1d5d70442ae None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.658002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.658851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 244/472] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:32.667644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.668186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.668667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.690945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:32.692362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:32.694165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 229/473] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:32 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:36:32.723045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:32.768890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:32.791313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 245/474] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:32.922256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9beb98-9de7-46df-abbf-ff0a020f27a8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:32.958692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:32.959730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:32.960221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:32.960683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:32.973669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:32.975498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:32.991771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:32.993116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:32.993958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:32.993958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:32.993958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:32.996171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 230/475] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:33.001606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:33.116745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:33.141955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c887-86bd-4daa-acba-7d45aa88e9f5 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:33.150317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:33.185606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:33.186635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:33.187208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:33.187749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:33.188596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:33.189107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:33.190965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:33.203735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 246/476] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:33.211852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:33.225445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac52a-13f9-4c63-a4b2-1d12b9a3e003 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:33.264015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:33.264015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:33.264015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:33.264015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:33.264368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:33.264368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:33.274834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.sql.core [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 03:36:33.274834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 03:36:33.274834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [parameters: {'id': '85e2af4275fe48f0af70ba646a9061b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 03:36:33.274834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93069) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result.read() Sep 03 03:36:33.291226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return ref.to_dict() Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:36:33.292124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application yield resource Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise self.value Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application session.commit() Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 03:36:33.293037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.session.flush() Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application compat.raise_( Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 03:36:33.293914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:36:33.294788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result.read() Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:36:33.296904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [parameters: {'id': '85e2af4275fe48f0af70ba646a9061b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:33.298929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 03:36:33.301039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.303112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 231/477] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Sep 3 03:36:33 2022] POST /identity/v3/roles => generated 144 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:33.303112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:33.325094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:33.411371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:33.417681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145f64a-2a5c-41f4-b70d-016b64ba6d5d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:33.436737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4827-546d-4d1b-ab38-a3548d91fac6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:33.454471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:33.455141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:33.455418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:33.455748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:33.468405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:33.473524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:33.474438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 03:36:33.475258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:33.479593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 232/478] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:36:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:33.481630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:33.507708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:33.508318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:33.508591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:33.508856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:33.509496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:33.510969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e98d2-f8d8-499c-bd25-f0f6570cfb79 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:33.543611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 247/479] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:33 2022] POST /identity/v3/users => generated 249 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:33.664494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9aed2c-c75b-47f2-8f3f-9bc227c94372 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:33.707941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:33.708636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:33.708915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:33.709183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:33.709850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:33.710149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:33.711566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb07d7e-16bd-4856-852b-cdd1bca84443 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:33.720310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 233/480] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:34.588974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791035fb-33e5-4680-bebf-2c146a7c55ed None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:34.589482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791035fb-33e5-4680-bebf-2c146a7c55ed None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:34.589482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791035fb-33e5-4680-bebf-2c146a7c55ed None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:34.591127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 248/481] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:34.596629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30043-57c3-450a-b264-4d42745ce663 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:34.597026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30043-57c3-450a-b264-4d42745ce663 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:34.597184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30043-57c3-450a-b264-4d42745ce663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:34.622107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d1b30043-57c3-450a-b264-4d42745ce663 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:34.623340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d1b30043-57c3-450a-b264-4d42745ce663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:34.636984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:34.694369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 234/482] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:34.759652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba30ee-9e98-4214-9f27-b28daf65afbd None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:34.759961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba30ee-9e98-4214-9f27-b28daf65afbd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:34.760192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba30ee-9e98-4214-9f27-b28daf65afbd None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:34.761807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 249/483] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:34.766843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959a210-6cad-48bd-aa08-6611a9402b89 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:34.767137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959a210-6cad-48bd-aa08-6611a9402b89 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:34.767413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959a210-6cad-48bd-aa08-6611a9402b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:34.789880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c959a210-6cad-48bd-aa08-6611a9402b89 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:34.791191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c959a210-6cad-48bd-aa08-6611a9402b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:34.857727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 235/484] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:35.032846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba30ee-9e98-4214-9f27-b28daf65afbd None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.069119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f6d4-e77f-4328-9953-a859d06b935c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.069452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f6d4-e77f-4328-9953-a859d06b935c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.069675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f6d4-e77f-4328-9953-a859d06b935c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.071318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 236/485] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:35.076106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.076385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.076659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.107187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:35.108430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:35.180970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.184405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba30ee-9e98-4214-9f27-b28daf65afbd None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.222445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.223489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.223768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.224038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.237126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:35.243352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:35.244293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.245066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:35.245669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 250/486] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:35.246636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.260769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 237/487] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:35.261680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.272023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.359559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7d848-4e10-460b-8148-bd26bd9c0b51 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.394459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.395296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.395590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.395867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.407112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:35.412508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:35.413401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:35.414199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 251/488] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:35.423674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.462943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d8151-f6bd-412b-9b9d-24a4bc0b1d50 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.471195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.498475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.499445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.499921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.500413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.501323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:35.501839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:35.503553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:35.512961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 238/489] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:35.518841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.593032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cc9f-f6b0-4fe7-b484-df9b6d14c716 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.608296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.632529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.633059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.633340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.633707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.634472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:35.634778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:35.636198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:35.639979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347e3ac-df3b-4163-961c-6671b25a3421 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.644896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 252/490] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:35.645767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.675350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.675986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.676261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.676529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.692638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:35.694379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:35.712696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:35.713635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:35.714479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:35.714479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:35.714479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:35.719037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 239/491] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:35.719908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.743181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.765562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fad0a-539a-45ea-8586-05b147860c27 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.812360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.813812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.814543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.815070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.829745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:35.833240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69a8a-9b5f-4236-b4a7-c2ddcfbf126e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:35.837582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:35.838469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:35.839286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:35.842405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 253/492] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:35.843549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.862429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:35.875896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:35.876524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:35.876798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:35.877178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:35.888803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:35.890269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:35.906879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:35.907806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:35.909286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:35.909286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:35.909286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:35.909286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 240/493] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:35.909286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:35.980245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3e0ba-160f-4a45-9cdf-52ea40edecad None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.014439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a1a-c99f-4465-8a10-0e7c1c74921d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.017260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.018238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.018708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.019200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.020048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.020545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.022236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.036100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 254/494] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.037296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.040319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.054772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.055419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.055696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.055968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.056642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.057014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.058351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.066878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 241/495] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.067765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.108240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.157292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566374a-8460-43af-83e4-efefc0bd16cb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.175355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d875cd6-ddf1-4ffd-a603-384e27f1b400 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.192562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.193172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.193448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.193716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.194349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.194648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.196127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.207475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 255/496] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.208244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.212590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.213221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.213494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.213760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] PATH_INFO: `/v3/users/nova` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.221460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.222953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-27253101-37fc-4527-99f0-97cf5cca5ebe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:36.235897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:36.237004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:36.238084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:36.238956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 242/497] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:36.240185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7957-1666-4f13-be27-933f1c9c6799 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.240471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7957-1666-4f13-be27-933f1c9c6799 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.240742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7957-1666-4f13-be27-933f1c9c6799 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.242168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 243/498] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:36.243287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968319a-96ce-4799-9f37-2ba66fb81086 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.243663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968319a-96ce-4799-9f37-2ba66fb81086 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.243978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968319a-96ce-4799-9f37-2ba66fb81086 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.245408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 244/499] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:36.245931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968319a-96ce-4799-9f37-2ba66fb81086 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.282216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.319336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8c074-8e2b-4dcb-bf02-6814a9f65812 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.340482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.347798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968319a-96ce-4799-9f37-2ba66fb81086 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.355343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.356274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.356549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.356816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.366557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.368024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.380588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.381213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.381484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.381753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:36.382674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:36.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:36.384477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:36.384477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:36.384477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:36.384477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 256/500] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:36.384477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.395919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.397371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-823704f6-e651-4c18-9bd1-7f01ae55f849 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:36.408848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:36.409904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:36.410832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 245/501] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:36.411498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fbfe9-6d26-4ae4-87db-5235c5bc5455 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.411498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fbfe9-6d26-4ae4-87db-5235c5bc5455 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.411794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fbfe9-6d26-4ae4-87db-5235c5bc5455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.430874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-605fbfe9-6d26-4ae4-87db-5235c5bc5455 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:36.432071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-605fbfe9-6d26-4ae4-87db-5235c5bc5455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:36.494702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf80f95-40ba-4883-ae42-3260795d9c09 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.499350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 246/502] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:36.500643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.500926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.501197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.515336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.520800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:36.521986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:36.531777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.532395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.532980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.533257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.534163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.534470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.536105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.576478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 257/503] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.576478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.598773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 247/504] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:36.600511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.621640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.710072 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdfa1c-a3fa-4e01-8293-dcccc3c8e339 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.712843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218e329-5c72-487b-adc7-14a95432ca7d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.748660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.749731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.750224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.750719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.751575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.751977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.753625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.754848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.755460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.755985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.756268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.756908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:36.757209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.758574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.771248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 258/505] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.771953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.800834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 248/506] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:36.801690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.816006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.875028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:36.890874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f10fae-f02a-43a7-8ba7-7277ffc186e8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.899439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d060-d248-4f9c-a618-75cb1e8f4d81 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:36.924356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.924956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.925233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.925500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.932871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:36.933489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:36.933766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:36.934032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:36.941046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.942512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:36.953682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:36.954577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 249/507] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:36.955546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:36.993897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:36.995298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.026370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.052362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16106bf-62c2-4384-8641-c8e51006db5b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.085926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.086548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.086845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.087127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.089134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.093723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 259/508] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:37.094527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.099588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.101076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:37.116682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:37.117607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:37.118639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:37.118639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.118639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:37.118639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 250/509] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:37.123172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.192424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b8ebb-b175-4da1-b92e-4b0d4ea92692 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.218470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3734-5d5e-4b52-938d-58909e3dc3c7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.220268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.226194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.226793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.227092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.227363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.258259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.258932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.259218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.259497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.269919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:37.276338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:37.277187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:37.277982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 251/510] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:37.278653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.294564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.296069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.323831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.377534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f46e98-2068-438c-8ddc-2c74522e9a01 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.394360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 260/511] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:37.395216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.412941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.413551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.413831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.414104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.414847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:37.415161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.416607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.423823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.433945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 252/512] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:37.434738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.493998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc82c-d0b0-45ad-9746-961ec8a2d94e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.527187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.527732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.528004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.528265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.532852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e7d51-9229-460e-9bdf-1ca6dc3c3fa3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.535172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.538100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.539579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:37.554515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:37.555469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:37.556536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:37.556536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.556536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:37.556536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 261/513] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:37.556536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.568011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.568625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.568905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.569182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.569825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:37.570129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.571543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.588498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 253/514] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:37.589299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.641574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.657830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2250f2-44c8-4e35-8c3a-a30cb2357b58 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.688544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246138-c63e-4819-8f78-40365d4f12cd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.690356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.691028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.691303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.691562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.692222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:37.692524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.693890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.701558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 262/515] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:37.702322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.723130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.723840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.724121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.724411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.734556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.736045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:37.750540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:37.751433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:37.752348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:37.752348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:37.752348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:37.752348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 254/516] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:37.752560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.769770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.809633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b9df0-145a-4014-a8a9-3d1669a9432c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.843651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.844267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.844545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.844820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.847730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a37b2-be76-49ad-98db-2d06e7656d2f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:37.859820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.880449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:37.881051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:37.881328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:37.881598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:37.882238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:37.882538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.883967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.896890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 255/517] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:37.897657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:37.915277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:37.916765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:37.970296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:37.993102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2eaf14-66d7-4993-8ee2-6ab1b1add532 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.014333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 263/518] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:38.015185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.026513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.027156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.027434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.027708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.028318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.029800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.051105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 256/519] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:37 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:38.051906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.063827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.126182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabddd1e-de74-4c41-a22f-bf44a792001a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.166230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.167353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.167812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.168263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.175841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0fe9b-867d-4066-a3e0-f3b6d88efb03 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.181093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.182907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:38.202096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:38.203076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:38.204083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:38.204083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:38.204083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:38.204387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 264/520] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:38.205133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.211326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.217960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.218987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.219325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.219602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.220604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:38.221073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.222748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.240037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 257/521] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:38.240723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.325627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.330012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dea59-f0f4-438e-897a-133a069754d9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.344532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50331443-29e4-45b9-8d4c-a0e0af5594aa None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.366692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.367950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.368224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.368494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.375497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.376950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.381012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.381894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.382386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.382872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:38.391127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:38.392018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:38.392905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:38.393324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 265/522] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:38.394043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.404808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.406557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:38.423447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:38.424666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:38.425652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:38.425652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:38.425652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:38.425652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 258/523] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:38.425652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.469274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.511693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49597d-a28d-467f-b7b0-0af39ccc027f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.526725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee98e84-8604-4996-ae06-a380099ea1fe None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.548210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.548954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.549328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.549668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.550381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:38.550750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.552275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.563020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.563729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.564074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.564411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.569059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 266/524] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:38.569059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.576002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.577483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.588032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:38.589760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:38.591032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:38.592074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:38.592651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 259/525] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:38.593428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.676108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a55-3934-451a-83c2-5e0515d4cbfd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.711005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ce6f-6940-4f90-a0f7-5784e6e6015a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.713100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.713834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.714201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.714303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.714632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.715442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:38.715764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.717223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.735019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 267/526] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:38.735834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.754688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.755594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.756082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.756486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.757266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:38.757702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.759391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.776332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 260/527] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:38.777387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.817062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:38.840126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3c293-bc9e-487f-bfa7-44d88c0e519f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.873359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.873981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.874263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.874542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.881949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.883463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.886496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9689-5395-4a1d-8781-c28f928be50d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:38.896863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:38.897837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 268/528] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:38.899542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.926062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:38.926659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:38.926966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:38.927233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:38.927918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:38.928215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:38.929747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:38.947608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:38.948228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:38.974177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.010113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713afe-70e2-40c5-951d-4879bfe85e10 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.043100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.043729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.044007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.044279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.045033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:39.045344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.046829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.052310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 269/530] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:39.053084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.055738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.067442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077854-1692-4553-8660-837a18f7b5e3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.107065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.107640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.107922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.108197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.116431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.118041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:39.131606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:39.132544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:39.133374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:39.137683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 262/531] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:39.138905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.196437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388645-f21f-420c-8670-f1c6f722af62 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.249047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.249822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.250234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.250687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.251693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:39.251999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.253508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.269884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.273056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 270/532] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:39.273882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.288399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3e21-b6a9-43f4-80ca-4feb10d1423f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.328017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.328625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.329149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.329424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.330155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=716e5d4773d643edbce5191347661f60) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:39.330491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.332672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.339982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 263/533] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/role_assignments?user.id=716e5d4773d643edbce5191347661f60&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:39.340619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.347346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.398699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2253-4bdd-4bcc-b8fc-8a11d747b65c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.437554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.438683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.439289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.439867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.440820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:39.441447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.443125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.457300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 271/534] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:39.463890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.464171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63bd62-856b-4416-bd6e-28d4097da731 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.508331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.508971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.509243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.509510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.551829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.643190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.644675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e63805-8242-443f-83b4-9321615a9cce None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.660225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039d2f-2797-4c8d-9a20-8ab4c5ae7b05 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.681907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:39.709478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.711387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.712500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.715244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:39.716593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=09249c2527c148cd99ebf899022f76fe) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:39.716593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:39.717845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:39.732310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 272/535] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/role_assignments?user.id=09249c2527c148cd99ebf899022f76fe&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:39.787326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 264/536] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:39.794844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:39.940734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1929b7-1dd7-47c4-b14e-69ad924e928b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:39.983456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:39.984457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:39.984982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:39.985844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.004998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:40.006782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9eb60328-a372-4d29-b8a1-8fa0ae5c1939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:40.027840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:40.028785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:40.029602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:40.029602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:40.029602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:40.035408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331e13-0cc4-4aa0-93dd-99fcbcf425b2 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.035960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331e13-0cc4-4aa0-93dd-99fcbcf425b2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.036420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331e13-0cc4-4aa0-93dd-99fcbcf425b2 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.037986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 265/537] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:40.038846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 273/538] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:40.043867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331e13-0cc4-4aa0-93dd-99fcbcf425b2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:40.049556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.050317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.050859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.086729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:40.087940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:40.139164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:40.151645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:40.228475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 274/539] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:40.242037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331e13-0cc4-4aa0-93dd-99fcbcf425b2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:40.277401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:40.278022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.278300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.278573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.279262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:40.279565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:40.280954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:40.298204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:40.298742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 266/540] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:40.473024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b984805-fdbf-49bc-aa78-370c6de5248e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:40.521611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:40.522159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.522430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.522694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.537327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:40.538787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:40.557092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:40.557983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:40.559444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 275/541] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:40.563979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:40.600416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:40.660750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:40.762609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2812cf-4f75-465b-831e-5a7c2d5841a1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:40.791239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:40.813093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:40.813700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.813977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.814246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.816364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:40.816832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:40.819610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:40.849057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8a5a-1e8d-4462-95c5-0bd5031d6847 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:40.850658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 267/542] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:40.858829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:40.944209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:40.944910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:40.945139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:40.945419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:40.964769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:40.969966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:40.970907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:40.971684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:40.987582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 276/543] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:40.995827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:41.002692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552a29-35ed-464b-a090-80df591c8bbd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:41.036472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:41.037090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.037368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.037637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.038367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=e5c0ba8119d54212a8c142353d27cd91) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:41.038677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:41.040179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:41.046239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 268/544] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/role_assignments?user.id=e5c0ba8119d54212a8c142353d27cd91&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:41.114536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:41.130070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7aa7c-c995-4188-96f6-66288765a428 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:41.167352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:41.167962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.168237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.168504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.169170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:41.169466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:41.170850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:41.180243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 277/545] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:41.191989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:41.225411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:41.328552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55fc8b-2ac3-4033-8677-5fe99417b02e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:41.372254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:41.372926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.373205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.373467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.395277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:41.396767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-d1570195-d359-4fd1-8a91-af165ae3d648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:41.413584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:41.414508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:41.415401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:41.415401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:41.415401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:41.415401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 269/546] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:41.421776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:41.548066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c665-8632-4901-871a-170a20b7b282 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:41.593256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:41.593795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.594068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.594334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.594995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:41.595290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:41.596711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:41.613971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 278/547] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:41.617216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbada2-a5b1-4f0b-aa90-ce4b5aafef3c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.617731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbada2-a5b1-4f0b-aa90-ce4b5aafef3c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.618215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbada2-a5b1-4f0b-aa90-ce4b5aafef3c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.619468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:41.620185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 270/548] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:41.625534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.626060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.626542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.635238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:41.663477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:41.665153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:41.738113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 271/549] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:41.779005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8902-468d-4201-ada2-074a39b826a7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:41.827287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:41.827923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.828196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.828462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.840157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:41.841638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-552f132b-6fd1-4221-8f91-5d45846126f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:41.856130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:41.857565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:41.858597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 279/550] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:41.862890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:41.874138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e008-d317-4628-8b5d-0bb33b88cf0d None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.874418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e008-d317-4628-8b5d-0bb33b88cf0d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.874689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e008-d317-4628-8b5d-0bb33b88cf0d None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.876316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 280/551] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:41.884252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3084-19ac-4595-a2ff-5b6722ad169b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:41.884528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3084-19ac-4595-a2ff-5b6722ad169b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:41.884797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3084-19ac-4595-a2ff-5b6722ad169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:41.888608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:41.904136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2a5b3084-19ac-4595-a2ff-5b6722ad169b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:41.905658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2a5b3084-19ac-4595-a2ff-5b6722ad169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:41.974916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:41.981558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 281/552] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:42.051895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3e10f-4983-4cdd-91e8-c590b66531b0 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.079457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77984f-0037-4c0e-a53a-4751d583bbc7 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.079749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77984f-0037-4c0e-a53a-4751d583bbc7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.080078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77984f-0037-4c0e-a53a-4751d583bbc7 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.081676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 282/553] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:42.086245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.086622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.087016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.105162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.106384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.106827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.107239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.108107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:42.108570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:42.109116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:42.110311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:42.110511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:42.150269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 272/554] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:36:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:42.150269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.213928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 283/555] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:42.214797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.243637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.303463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600a14-5209-4135-aa55-1892cbeff17e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.347405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.348496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.348777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.349051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.364463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:42.370710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:42.371625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.372453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:42.373388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 273/556] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:42.373527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.375609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f504aa-81bf-4c7c-b7ff-bbc762705211 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.409968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.421810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.422737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.423708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.424260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.491349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61448c-a94e-4ef5-bbfa-dbaad687ab40 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.511368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:42.512862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:42.523140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.529720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.530602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.530914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.531320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.542051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:42.546716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:42.547633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.548431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:42.552719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 274/557] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:42.553312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.618410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 284/558] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:42.619115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.652227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.662205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa0fd-08ea-416f-94bc-c978f2c71cc1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.703875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.705244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.705743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.706861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.708311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:42.708750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:42.710340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:42.719889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 275/559] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:42.720974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.734398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2929-72a9-4b59-872f-2d6ec4786b17 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.781467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.782100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.782379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.782639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.791946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.792958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:42.797834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:42.799140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 285/560] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:42.799988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.847938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bbbf-0eca-40d5-bdef-04d6c8f2569c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.885634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.886242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.886985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.887268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.887938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:42.888237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:42.890284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:42.898918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 276/561] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:42.900320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:42.903943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:42.913802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22cdf-eff5-4086-8aa2-ffd5299520f5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:42.948557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:42.949192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:42.949470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:42.949740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:42.960317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:42.961774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:42.978060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:42.979677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:42.980605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:42.980605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:42.980605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:42.980605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 286/562] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:42.981137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.004816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206fae-17aa-4a65-a4f6-edeb5aff1de5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.021402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.039771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.040404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.040677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.040940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.052112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.053733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:43.067879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.069026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:43.069994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:43.069994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.069994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:43.071371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 277/563] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.072429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.100773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ba5b7-fad9-4081-b326-cebc6856b470 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.136964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.137488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.137775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.138019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.138670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:43.138995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.140381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.150344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 287/564] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:43.151166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.153783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.191323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a4d93-f1a0-40e2-a94a-9f29f973ee96 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.223880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.224499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.224774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.225039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.235836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.237349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:43.251748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.252881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:43.258432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:43.258432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.258432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:43.258432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 278/565] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.258792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7db3-9248-404e-93d8-811bd92f94d5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.258912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.275910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.291591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.292207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.292486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.292760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.293396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:43.293708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.295120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.307520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 288/566] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:43.308471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.353540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb41f-07df-4da4-8c74-90ac95bb8eb5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.367826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.388943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.389489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.389762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.390030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.390677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:43.391006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.392419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6b2d-8235-42da-95e0-550e247bc901 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.404950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 279/567] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:43.409773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e838-7321-493d-887a-d6681fd83ee9 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.409773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e838-7321-493d-887a-d6681fd83ee9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.409773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e838-7321-493d-887a-d6681fd83ee9 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.409993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2818f-b4f2-4f85-8264-53963c5d7001 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.410082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 280/568] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:43.410082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e838-7321-493d-887a-d6681fd83ee9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.442675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.443327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.443599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.443862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.453457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.454931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:43.468860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.469975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:43.470876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:43.470876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:43.470876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:43.470876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 289/569] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.471117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.497188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.518348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e838-7321-493d-887a-d6681fd83ee9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.551998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.552583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.552876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.553139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.553782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:43.554080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.555492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.566519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 281/570] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:43.567306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.571126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e548-5a2c-40ff-b2cc-546ece6f0db5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.584811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.605016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.606035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.606308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.606574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.613538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.615016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-2b79f629-6366-4bab-a619-02c1e83389b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:43.626387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.627503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:43.628369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 290/571] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.629163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.629437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.629708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.646964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:43.648098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:43.672474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb2d04-0906-4fee-901d-4ccb20a2a039 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.706403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.707055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.707336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.707606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] PATH_INFO: `/v3/users/glance` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.714775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.715630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 291/572] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:43.716233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.716452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.730227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:43.740619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:43.742496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 282/573] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.743650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.821189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719b82c-37ab-4308-8ede-a3dcec290943 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.823443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:43.849069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db851-4bfc-4897-adc9-e7d53442f68a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.854724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.855356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.855930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.856205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.856841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:43.857139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.858520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.869495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 292/574] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:43.870271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.891449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:43.893181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:43.893916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:43.894361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:43.903956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:43.905994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:43.921135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:43.922073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:43.923078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:43.927239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 283/575] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:43.927961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:43.977159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162c254-4116-4f4d-9c95-d107a32810d1 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:43.988775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.010706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.011340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.011614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.011877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.012554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.012852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.014277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.026178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 293/576] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.026985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.029338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f4e-22be-4ecb-a2c7-091d8c378769 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.053081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.064203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.064808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.065087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.065359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.066081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.066386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.067856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.085165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 284/577] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.085961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.130283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26550ab3-70d6-4e58-9efc-86886e15cc1a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.165400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.165933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.166206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.166470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.173232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.174744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.182168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676646-cd5b-406f-a0e2-3842798937b9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:44.185918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:44.186888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:44.187838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 294/578] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:44.188424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.199353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.217910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.218521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.218794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.219097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.219820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.220119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.221574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.239432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 285/579] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.240239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.274416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.290962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8ee4f-9760-406d-9376-a01045277763 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.323013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.323634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.323911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.324185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.334899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:44.338349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151579-e3f5-4bdd-b26e-63df631c486e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:44.340058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:44.340996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 295/580] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:44.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.375668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.376574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.376862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.377176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.377898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=323c45a05bd241458486b8919c93dd2e) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.378202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.379676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.384919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 286/581] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/role_assignments?user.id=323c45a05bd241458486b8919c93dd2e&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.385689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.435035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d6b4-ef4e-4eb2-a423-846ab500c644 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.498911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.499569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.499858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.500124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.509537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c049999-2a76-4620-8437-1a06fd7b6e3c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.512279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.513751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.517045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:44.530479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:44.531692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:44.532561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:44.532561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.532561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:44.532561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 296/582] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:44.532561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.551263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.551869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.552147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.552420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.553160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.553464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.554935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.572482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 287/583] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.573324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.644659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4576-8079-4837-8f73-921b8f3e458e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.675472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.680450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79de3c-ea0c-498d-8b01-81a8475a34d4 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.688206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.689223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.689654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.690060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.701324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.703085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.718335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.719098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:44.719348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:44.720265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:44.725997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:44.725997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.725997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:44.726179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.726179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.726179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.726179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.726443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 297/584] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:44.726443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.726566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.738855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 288/585] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.738855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.783202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:44.841643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97917-c6b5-4b4c-94e3-e93e1e5dbf35 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.850740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f596e-70cb-4e9b-b5b1-21d27f5805ce None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:44.881028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.881658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.882017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.882285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.888389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:44.889001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:44.889281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:44.889554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:44.890349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:44.890658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.892064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.895406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:44.896866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:44.906407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 298/586] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:44.907298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:44.915886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:44.916814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:44.917677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:44.917677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:44.917677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:44.917677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 289/587] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:44.917903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:44.937870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.021868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67a4e9-81bf-4642-86d4-1e1d203ddb5f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.030699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff2c0f-a751-4fd2-a540-710335df960a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.032381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.059556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.060175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.060450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.060727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.061374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.061684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.063096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.065786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.066627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.067234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.067686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.074677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 299/588] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.075499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.083683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.085383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:45.100318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:45.101271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:45.102148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:45.102148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:45.102148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:45.102148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 290/589] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:45.102368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.168977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.186423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf2d6-b48e-4a33-8809-48902555e16a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.206870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e61a5-90df-4e07-a345-8d5f2c66329a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.227224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.227861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.228136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.228409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.239354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.240878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.241121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.241753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.242014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.242284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.242964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.243271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.244630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:45.254040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:45.255814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 300/590] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:45.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.264677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 291/591] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.265099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.288785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.365176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f8c7-91de-41bf-95d8-0bb0f8b31786 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.366826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74db73-695e-4693-8d1b-65863276a944 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.392628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.401324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.401964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.402242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.402516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.403773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.404381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.404655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.404922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.405569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.405871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.407417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.409694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.411178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.420670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.421813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:45.423344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:45.424246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.425124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:45.429089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 301/593] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:45.429930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.531123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.543573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514309b1-c1c2-4013-8a7e-ccd9803e23ff None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.544677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857cf6-2c8c-407e-a8b3-646fbb0f58ec None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.582313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.582954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.583234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.583503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.583809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.584196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.584499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.585090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.585561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.585931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.586237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.593591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.595414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.597997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 293/594] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.599145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:45.607884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:45.608761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:45.609679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:45.616565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:45.617624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.652785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.710255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5eaf6-6429-4c37-8c02-c050168f80f3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.725595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6661f-231a-4fd2-aa26-ff2ac6e9e571 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.745960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.746593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.746918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.747191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.755491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.757084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.764398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.765057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.765286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.765557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.766239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.766541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.768101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:45.770957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:45.771859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 294/596] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:45.778233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.787919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 303/597] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.787919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.797624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.883555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b129-617b-48b1-8262-d8ab69e40b00 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.886740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:45.892649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24b7a9-fbef-46f2-aad3-1bd368680a13 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:45.919990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.920657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.920936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.921223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/716e5d4773d643edbce5191347661f60/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.928970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:45.929580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:45.929856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:45.930502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:45.931257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:45.931565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.933003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.937667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=716e5d4773d643edbce5191347661f60, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:45.939142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:45.945764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 304/598] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:45.946857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:45.956814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 295/599] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:45 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/716e5d4773d643edbce5191347661f60/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:45.967851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.025066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:46.063749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc520d-e8da-4918-af23-f533ab406f30 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.105366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.106423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.106777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.107087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.107888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:46.108388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.109815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.118214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6be1b-b86c-481a-99c2-0496b6d35974 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.131714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 305/600] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Sep 3 03:36:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:46.132762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.172300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.174533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.175098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.175577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.194953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=8e0e6be0fc61491fa25193834b4fa8af, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.197422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.216406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 296/601] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:45 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:46.217715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.222470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:46.259613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b810f-5143-4cd4-b280-d937fc7e4fc2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.290691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:46.301639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.302787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.303293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.303831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/09249c2527c148cd99ebf899022f76fe/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.321842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=09249c2527c148cd99ebf899022f76fe, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.323672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c65efc-c959-4045-bc2b-b82451e263d3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.341722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 306/602] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:46 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/09249c2527c148cd99ebf899022f76fe/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:46.362246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117bbcc-e4aa-41ac-a402-8e213afff8b9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.411303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.412461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.413352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.413352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.422630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f80af-ab02-42f4-9a43-74cff62f369a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.423011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f80af-ab02-42f4-9a43-74cff62f369a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.423251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f80af-ab02-42f4-9a43-74cff62f369a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.424940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 307/603] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:46.430168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.431281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.431801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.431907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.432368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:46.451992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:46.453018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:46.453898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:46.453898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:46.453898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:46.453898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 297/604] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:46.454950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:46.456775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:46.460202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.549293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:46.565325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 308/605] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:46.597607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2cdd-4480-48b4-a7a4-2ffea0eadf64 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.632707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.633323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.633602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.633877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.634527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:46.634915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.636366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.649431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 298/606] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:46.654856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.796224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd4c01-4bdb-4c69-a1d9-cf99b2de7855 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:46.807584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:46.848613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:46.849170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:46.849448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:46.849718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:46.851638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.865001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:46.866881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:46.885191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:46.886132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:46.887061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:46.895725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 309/607] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:46.897677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:46.999773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba277c6-2828-45ec-82ba-de3c1d93cd93 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.022641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:47.054796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.055452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.055731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.056003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.074880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:47.079678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:47.080656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:47.081494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:47.082118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d17bc-cc0f-4214-9187-de9961d2ed1e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.082514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 299/608] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:47.087261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:47.129175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.129974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.130685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.131370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.132184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:47.132501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.133942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.148779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 310/609] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:47.154634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:47.201969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:47.252078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be188-80d6-4062-bc7e-c435a965afb8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.296170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.296811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.297094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.297371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.298053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:47.298361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.299858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.317723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 300/610] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:47.320623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfd46d-a020-4288-a450-3a9b56dc2072 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.323199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:47.356400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.357061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.357337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.357603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/e5c0ba8119d54212a8c142353d27cd91/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.372215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=e5c0ba8119d54212a8c142353d27cd91, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.373722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.373830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:47.406154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 311/611] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:47 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/e5c0ba8119d54212a8c142353d27cd91/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:47.456263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae85f2-8f36-449a-8851-87a1e4393bf2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.494565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.495234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.495524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.495791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.507786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.509390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:47.525592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:47.526589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:47.527512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:47.527512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:47.527512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:47.527512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 301/612] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:47.536085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:47.641516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:47.668520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d5025-315d-4e44-b0a6-d309a04341da None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.704525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.705243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.705522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.705794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.706434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:47.706737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.708180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.721108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 312/613] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:47.726525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:47.869506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:47.879507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfbe-79f9-4f7d-940c-25421a33b470 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:47.932988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:47.933556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:47.933835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:47.934108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:47.947900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:47.949386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:47.964346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:47.965312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:47.966225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 302/614] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:47.971540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.122851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18b93-fb9c-4567-b53c-cb3d4b592351 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:48.168704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:48.169365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.169648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.170164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.170643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:48.170984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:48.172419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5d3a0-1833-45a6-be99-f73c2fac29aa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:48.175016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:48.194590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 313/615] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:36:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:48.208138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.354058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e599c-c5e2-41aa-9c75-44d62fe3a1b2 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.354360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e599c-c5e2-41aa-9c75-44d62fe3a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.354615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e599c-c5e2-41aa-9c75-44d62fe3a1b2 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.356607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 314/616] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:48.361261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.361545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.361818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.375771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad279-3af5-4614-9173-d828ad2697a1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:48.383661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:48.385090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:48.444546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:48.445349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.445461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.445721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.456755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 315/617] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:48.467943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:48.469679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-652e8e71-e37d-4d28-82c2-7cff00421e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:48.499549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:48.500635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:48.501524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:48.501524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:48.501524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:48.506280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 303/618] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:48.512161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.628962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b54d-c643-40b4-bd62-52d22f495fa0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:48.648677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6d2e-6704-4a20-a1c8-d250909a7fbc None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.649322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6d2e-6704-4a20-a1c8-d250909a7fbc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.649798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6d2e-6704-4a20-a1c8-d250909a7fbc None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.651382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 304/619] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:48.656651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.656937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.657213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.663179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:48.683285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:48.684400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:48.685140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.685408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.685883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.686539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:48.686865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:48.688342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:48.690670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:48.691908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:48.711547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 316/620] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:48.716679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.822981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 305/621] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:48.823915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.869852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db31da-cc13-4a5d-837c-8990eef23990 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:48.906227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:48.906891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:48.907173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:48.907539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:48.915046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:48.916668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:48.928820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:48.929758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:48.930658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 317/622] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:48.936090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:48.955435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6976df-17c9-48f3-9638-c8bf73ebf9d7 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:48.985071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.016440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.032455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.033209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.033486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.033757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.045103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aad88a-96f6-4486-b473-bf369ec83bcc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.063711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:49.068818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:49.069748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.070623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:49.076789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 306/623] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:49.083859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.092155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.093076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.093572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.094286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.095217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:49.095790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.097457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7498f0-aae3-48d0-9e33-3571692ce8f2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.111983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 318/624] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:49.118160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267df75-e025-4116-8490-e3dc9e73ec4a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.118789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267df75-e025-4116-8490-e3dc9e73ec4a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.119292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267df75-e025-4116-8490-e3dc9e73ec4a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.121203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 319/625] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:49.122112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267df75-e025-4116-8490-e3dc9e73ec4a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.197285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4742-7406-4da7-b9c5-4e91589a9e25 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.230893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.231530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.231789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.232041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.232705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:49.233007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.234406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161a99c-7b99-43f2-8815-40648d148ae6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.240986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267df75-e025-4116-8490-e3dc9e73ec4a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.244930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 307/626] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:49.245064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.246038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.246393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.246715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.270869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:49.272078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:49.281171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.282004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.282143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.282370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/323c45a05bd241458486b8919c93dd2e/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.297681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=323c45a05bd241458486b8919c93dd2e, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.299311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.313674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 320/627] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:36:49 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/323c45a05bd241458486b8919c93dd2e/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:36:49.314782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.337915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.370639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 308/628] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:49.372158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.464473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db833-c3dd-4b51-9bf4-ac62dfbb7feb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.527271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371962ec-8458-4f83-ac25-9c2a7bc25873 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.527464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.528011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.528305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.528826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.544701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:49.551146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:49.552150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.553067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:49.553695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 321/629] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:49.559895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.568174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.568820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.569097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.569368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.580358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.586603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.588743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:49.612893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:49.615094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:49.617264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:49.617264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:49.617264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:49.617264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 309/630] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:49.624017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.698849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822460-d094-4803-84cd-aaff40fd4184 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.699151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.740878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.741492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.741783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.742053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.742724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:49.743064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.744461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36579232-2f16-4d47-91a6-5d334cd0dd61 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.752889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 322/631] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:49.754239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829081-f675-4e39-9109-34cef6ee7512 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.754536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829081-f675-4e39-9109-34cef6ee7512 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.754833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829081-f675-4e39-9109-34cef6ee7512 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.756254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 323/632] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:49.756755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829081-f675-4e39-9109-34cef6ee7512 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.771778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4252c-1d80-4b4b-a6e8-055cab264aff None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.806325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.806934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.807145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.807417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.808066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:49.808376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:49.809763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:49.822275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 310/633] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:49.823440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:49.847400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.874640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829081-f675-4e39-9109-34cef6ee7512 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.909806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.910446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.910729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.911029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.921453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-402f6fef-5d66-4c42-8c5c-ddf1d04fad86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:49.926330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:49.927325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:49.928146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 324/634] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:49.929345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.929345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.929345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:49.930301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:49.938488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552c81-a44b-4282-8a42-1e98f63f8e3d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:49.953787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:49.955121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:49.983959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:49.985261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:49.985903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:49.986401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.001727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.004421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:50.024325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:50.025270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:50.026135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:50.026135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:50.026135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:50.026135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 311/635] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:50.026364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.031737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 325/636] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:50.032589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.116161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.155162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c1a7-485b-468b-85d3-bb7dc643a75a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.159918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da379ac-03c1-48cf-af62-13aace3ba1a3 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.199937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.200565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.200843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.201116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.201672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f759-820c-4225-b82f-d583512272dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.202528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f759-820c-4225-b82f-d583512272dd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.203137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f759-820c-4225-b82f-d583512272dd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.203602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f759-820c-4225-b82f-d583512272dd None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.205032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971f759-820c-4225-b82f-d583512272dd None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:50.205515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971f759-820c-4225-b82f-d583512272dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.207094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971f759-820c-4225-b82f-d583512272dd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.209591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.211505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.217446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 326/637] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:50.218459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f759-820c-4225-b82f-d583512272dd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:50.231543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:50.232841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 03:36:50.233904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:50.237838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 312/638] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:50.239169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.253185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.341564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f759-820c-4225-b82f-d583512272dd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.355373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbfe1c-d8dc-418a-814d-e699c038c7f3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.370334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.382429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.383067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.383346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.383874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.384256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:50.384556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.385931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.391807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.392473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.392753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.393030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.403149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 327/639] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:50.404019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.404273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.405737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:50.420970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:50.421944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:50.423352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:50.423352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:50.423352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:50.423352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 313/640] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:50.423352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.485006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.515848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113778-9dcd-4e07-80ad-0dcd03f2accc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.530478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52f3ee-5d6a-4dd4-be07-ecd4eb2b6b4f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.551716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.552333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.552614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.552887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.553613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:50.553929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.555418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.569592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.571447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.571933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.572690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.574485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 328/641] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:50.575389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.590627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:50.598409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:50.599535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:50.600604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:50.606056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 314/642] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:50.606771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.629468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.685912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4e78c-6fe6-4d6a-bcc7-fd5e52eb2149 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.712291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e60f7-730a-4d89-a8c9-989138afc047 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.720841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.721647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.722123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.722538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.729712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.731640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.732902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:50.743833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:50.744794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 329/643] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:50.745900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.754340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.754987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.755269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.755548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.756204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:50.756509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.757891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.774956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 315/644] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:50.774956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.852520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e927685-7302-410b-bf0a-682753d137c0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.861279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.877521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaa0dd-dd6a-4694-81c2-5577ec6f7dca None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:50.896338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.897106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.897411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.897807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.914791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:50.915472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:50.915697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:50.915972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:50.916638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:50.916954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.918334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:50.926900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 316/645] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:50.927692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:50.973422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:50.973817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:50.975269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.034421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d8066-0eb2-4c8a-a119-dc7ccd55e134 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.069884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.070568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.070881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.071159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.071892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:51.072200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.073658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.078082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 330/646] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:51.079716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.090463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.091279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 317/647] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:36:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:51.095813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.187713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8b48-9d38-4118-b87c-bf525fcd04db None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.194720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.197842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3f50-181c-4c09-bde5-a48f5a3ef420 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.225249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.225883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.226311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.226587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.235165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.235647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.236562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.236848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.236951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.237149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.248612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:51.249034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:51.249942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:36:51.250910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:51.251484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.255079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 331/648] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:51.255955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:51.271740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:51.272748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:51.273614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:51.273614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.273614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:51.273614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 318/649] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:51.273614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.335601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.374067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f691-5262-4a1d-8584-cd095e0ac5d2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.378433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fcc91-0c4f-4a51-95e3-4cc9e9b36108 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.408183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.408821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.409101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.409375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.411916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.412528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.412807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.413079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.420249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.421713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.433754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:51.436318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:51.437267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:51.438206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:51.438206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.438206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:51.438206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 332/650] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:51.438206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.483842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.485302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.534618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0408d51-f4e0-4dee-afe3-747f6d452850 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.550895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.567887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.568494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.568776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.569048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.569779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:51.570091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.571540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1747ed-7a68-4aee-b1dc-6274ecf298c1 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.581749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 319/651] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:51.582579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 333/652] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23c5eb-77e0-48fe-b079-7c70bdaf6083 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23c5eb-77e0-48fe-b079-7c70bdaf6083 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23c5eb-77e0-48fe-b079-7c70bdaf6083 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 334/653] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:51.594117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23c5eb-77e0-48fe-b079-7c70bdaf6083 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.638764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.688410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d3efb-2e62-44bb-9923-0a307ff465ca None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.693555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23c5eb-77e0-48fe-b079-7c70bdaf6083 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.721725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.722336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.722611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.722931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.723572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:51.723872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.725266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.726648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.727284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.727570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.727844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.728481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:51.728786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.730233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.736630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 320/654] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:51.737432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.746396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 335/655] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:51.747245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:51.785579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.834259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecb487-e642-44ee-b196-a6d616d86f54 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.846405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:51.849054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04cec4-52ba-4226-996d-0ed08ba51b0d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:51.867708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.868348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.868630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.868911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.879754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.881251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.884635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:51.885668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.886155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.886659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2fe86b80-4939-4028-a435-6c6e00449647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:51.901317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:51.902414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:51.903777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:51.903777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:51.903777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:51.903777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 321/656] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:51.903777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:51.904116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:51.904116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:51.922578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:51.923882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:51.961050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:51.962434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:51.989773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 322/657] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:51.990592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.004215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.069795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 336/658] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:52.075014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.086227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.092846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b974b-37b4-4fad-bcbe-0e9806b0ddf0 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.133701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.134363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.134636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.134937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.142571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.144063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:52.155406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:52.156321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 323/659] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:52.158214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.182460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08088-2f42-4968-bdda-cf53a2f03adc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.219928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.220521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.220801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.221070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.225096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.230095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.232058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:52.244134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:52.245023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:52.245917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:52.250058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 337/660] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:52.250915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.281293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc72a9-cf5d-4746-893d-c06952bbbc45 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.313234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.313842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.314117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.314393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.315071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:52.315360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.316779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.333656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 324/661] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:52.333656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.345184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.350526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7645a-8f1b-4f80-9964-1a00a3930c0b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.384163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.384816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.385092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.385362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.395807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.397265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:52.410995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:52.411945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:52.412849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:52.412849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:52.412849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:52.412849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 338/662] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:52.413134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.430343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b1f6-4031-434b-808d-29d34338b70d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.463006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.464966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.465608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.465841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.466110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.467432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:52.467958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.469721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.490124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 325/663] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:52.490899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.521912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b0981-b5d2-4862-895e-18e5d54c56b3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.556181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.556785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.557051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.557333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.558045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:52.558347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.559839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.565979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.571773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 339/664] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:52.572566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.599559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b47d-85d0-42c4-9f84-1f64756a7440 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.633552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.634184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.634460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.634726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.645228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:52.649910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:52.650881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 326/665] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:52.651741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.671319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b358e-8a02-405b-94c0-22db9451413a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.680414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.705207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.705839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.706116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.706487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.713269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.714711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:52.725583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:52.726609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:52.727604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:52.728193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 340/666] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:52.728993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.762939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e745ad-4dca-4607-b402-cd65a1aed5d6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.791034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.796303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.796911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.797193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.797445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.798090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:52.798420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.799861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.810885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 327/667] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:52.811678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:52.831190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797c786-5f64-4b86-bf96-b5c3a7e339e9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.864466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.865073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.865351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.865622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.899563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:52.910317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17006-3803-4796-a1d7-022234fd18fd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:52.932429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.933884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.945733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:52.946341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:52.946614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:52.946916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:52.947632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=716e5d4773d643edbce5191347661f60) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:52.947961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:52.949371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:52.955098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 328/668] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/role_assignments?user.id=716e5d4773d643edbce5191347661f60&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:52.955853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.028904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.056894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 341/669] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:36:53.062412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.081359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829be42-4e29-4121-bef4-4bdb68f059e6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.122457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.123118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.123396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.123667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.124330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.125103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.126764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.137492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 329/670] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.138320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.140085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.183070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e46fc7-60da-46c8-85d5-88daab93e9cf None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.218461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.219090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.219366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.219631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.220336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.220636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.222060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.238150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 342/671] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.239900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.261160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b2e28-10b2-44eb-8346-b80c6c5a4f0b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.278188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.296473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.297073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.297505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.297609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.304886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.306352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:53.317973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:53.318922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 330/672] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:53.319792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.353525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c28-ef4c-4b80-8157-1aa043db0166 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.380167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.389232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.389890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.390127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.390396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.402178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.404334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:53.419546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:53.420692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:53.421589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:53.421589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:53.421589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:53.421589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 343/673] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:53.421863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.431178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd8ff-dafe-4f22-a273-87c5c3d0462a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.467874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.468878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.469367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.469782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.481662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.483521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:53.498493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:53.499705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:53.500814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:53.500814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:53.500814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:53.502532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 331/674] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:53.503476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.528888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989debd9-13f1-4a39-905b-95a884c104f8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.537528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.564651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.565211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.565514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.566028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.566543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.566938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.568302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.574641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 344/675] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.575279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.628548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.637670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a92d6-05e6-4e78-a8f3-c7a37b4879d6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.679892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.680604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.680801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.681074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.681790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.682098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.683639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.698050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 332/676] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.699303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.706453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fc939-35d6-4e21-bab8-a516b14f86bc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.750468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.751452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.751737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.751988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.752625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.752923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.754287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.772174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 345/677] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.773817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.809147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.826134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a10585-262c-4f13-a4c2-4ad39681fc16 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.867242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.867858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.868133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.871057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.871951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.872474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.874083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.889651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 333/678] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.890936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:53.905156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6928b1-54dd-4a5f-b68c-39d7737a8ff4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:53.908764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:53.944607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:53.945645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:53.946163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:53.946693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:53.947644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=09249c2527c148cd99ebf899022f76fe) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:53.948151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:53.949815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:53.956696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 346/679] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/role_assignments?user.id=09249c2527c148cd99ebf899022f76fe&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:53.957746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.018046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d622af-c1c4-4130-8b7c-7ff079115cfc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.070127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.071514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.073175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.075203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.087715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.090941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:54.112438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:54.113442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:54.114355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 334/680] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:54.119478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f024c-d95f-4564-9e6f-835ae4767f7e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.119650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.146652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.176955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.177592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.177867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.178136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.189304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.190758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:54.203368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:54.204339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:54.205354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 347/681] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:54.210316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.247213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.264303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e7a1-a766-45af-91b8-23e08196ac93 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.306486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.307617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.307816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.308052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.309531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:54.310035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.311508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.329944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 335/682] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:54.330916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.331174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12695469-f555-4fe4-bfda-9efd5776144d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.380305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.381270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.381956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.382373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.383516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:54.383962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.386282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.409176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 348/683] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:36:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:54.415467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.418030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.456490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056fa5b-e5d6-4138-b232-9c285897168f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.492905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.493512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.493784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.494049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.494880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=e5c0ba8119d54212a8c142353d27cd91) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:54.495191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.496733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.503295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 336/684] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:54 2022] GET /identity/v3/role_assignments?user.id=e5c0ba8119d54212a8c142353d27cd91&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:54.566750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.583657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1cf9-e5d6-489c-9792-a0aa9a216cd2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.632820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.633359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.633672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.633947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.713321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.714879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:54.804170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.821376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 349/685] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:36:54.827151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:54.933167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e966e-5cc1-488e-9be4-856b78b59acb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:54.974170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:54.974825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:54.975119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:54.975387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:54.977279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:54.987454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:54.988911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-9ffc4842-5635-493b-8b8d-035835624883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:55.004476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:55.005452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:55.006352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:55.006352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:55.006352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:55.006352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 337/686] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:55.011482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:55.099090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8b9f-ceec-4155-b419-64e4e93eff87 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.099443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8b9f-ceec-4155-b419-64e4e93eff87 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.099640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8b9f-ceec-4155-b419-64e4e93eff87 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.101260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 338/687] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:36:55.114475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258915-9c69-4891-b21e-3e001917afca None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.114757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258915-9c69-4891-b21e-3e001917afca None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.115050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258915-9c69-4891-b21e-3e001917afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.143403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db930-ca6e-429d-a6bb-f5f18a4757b0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:55.147286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-18258915-9c69-4891-b21e-3e001917afca None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:55.152624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-18258915-9c69-4891-b21e-3e001917afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:55.224090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:55.224720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.225003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.225278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.225929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:55.226238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:55.227672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:55.252646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 339/688] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 03:36:55.259379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 350/689] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:55.261160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258915-9c69-4891-b21e-3e001917afca None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:55.421380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258915-9c69-4891-b21e-3e001917afca None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:55.449029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:55.468256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:55.469237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.469725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.470236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.477322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:55.479018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:55.491079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:55.492077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:55.493025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:55.495456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 340/690] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:55.503975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:55.581427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:55.653519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cf91c-469b-4120-a873-7644192922a5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:55.701335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:55.701954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.702234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.702505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.703251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:55.703556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:55.705022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:55.721757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 351/691] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:55.727879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:55.743450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:55.840724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df43d-e10a-4c96-97db-1ba60020de3f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:55.882614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:55.883642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.884141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.884973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.885881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=323c45a05bd241458486b8919c93dd2e) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:55.886713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:55.888934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352375e-b6d9-4560-ac48-bf6a9fd1b5b8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:55.896553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 341/692] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:55 2022] GET /identity/v3/role_assignments?user.id=323c45a05bd241458486b8919c93dd2e&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:55.898517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef93a-4232-41c3-abde-b4720bb38685 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.899118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef93a-4232-41c3-abde-b4720bb38685 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.899634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef93a-4232-41c3-abde-b4720bb38685 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.902152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 342/693] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:55.907917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:55.908743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:55.909198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:55.910517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484567b-be27-4e3f-aab1-cf414dc522e0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:55.919087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:55.931699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:55.933464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:55.980005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:56.010197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 343/694] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:56.031595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.032389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.032778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.033120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.053033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:56.054613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:56.079487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:56.080725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:56.081822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:56.081822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:56.081822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:56.085376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 352/695] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:56.086762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.193916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e7e63-8730-41c2-aced-1961a98bf035 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:56.233931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.233931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.233931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.233931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.234372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:56.234372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:56.235597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:56.250865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 344/696] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:56.254063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.304045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.342875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:56.371734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db421bf-5703-400a-af06-9445d50aac32 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:56.412424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.413201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.413529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.413761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.414734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:56.416647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:56.433646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3e31d-c48c-486d-ade6-699601a966ba None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:56.441707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 353/697] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:36:56 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:36:56.477461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.478117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.478397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.478669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.492946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5c86f1a0-614e-4bca-b29a-dab6bccc10a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:56.499504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:56.500567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:56.501527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 345/698] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:56.507227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.509500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aebcf8-50dc-480e-b7b3-8c27f5fa9ba4 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.509743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aebcf8-50dc-480e-b7b3-8c27f5fa9ba4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.510016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aebcf8-50dc-480e-b7b3-8c27f5fa9ba4 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.513278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 346/699] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:56.526738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.527335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.527668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.531995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:56.554544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:56.556480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:56.633136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 347/700] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:56.652187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dafb4-1555-49f3-8dda-dc025711901a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:56.688327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.688986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.689295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.689540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.690209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:56.690511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:56.691936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:56.700917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 354/701] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:56.706208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.729300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:56.811975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8d0c1-9912-4790-a938-c5cf2f4a8b0a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:56.849329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:56.849968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:56.850248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:56.850520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:56.861876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:56.863380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:56.879397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:56.880400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:56.881295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:56.881295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:56.881295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:56.881295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 348/702] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:56.886366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:56.996373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.019972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.024358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50489b-91b5-478e-80d2-fb66b3f9d270 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.061350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.061962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.062289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.062501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.063162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:57.063466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.064960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.088910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 355/703] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:57.094668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.174013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.214517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8d716-e083-4f62-9fc6-430a6ccfe4a6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.224813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7a8e-daee-4cfe-af5d-5ecee79f86d8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.255853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.256871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.257153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.257425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.258010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.260070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.269025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.269914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.270439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.270954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-41077697-37ca-476d-90d2-d26648549b22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:57.274096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:57.275144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.275986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 349/704] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:36:57.276807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186aa1e-6313-4a7e-a6b0-e867b082f759 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.277398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186aa1e-6313-4a7e-a6b0-e867b082f759 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.277398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186aa1e-6313-4a7e-a6b0-e867b082f759 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.279169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.282663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 350/705] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:57.283556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186aa1e-6313-4a7e-a6b0-e867b082f759 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.295129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.296755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1f5d4b5e-ab75-43d2-b10d-fc54436c6144 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:36:57.309803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:57.310771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:57.311743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 356/706] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:57.312881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.312881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.313048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.331939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:57.333146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:57.403223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186aa1e-6313-4a7e-a6b0-e867b082f759 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.404121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 357/707] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:57.405267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.451821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.452493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.458837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.458837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.458837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:57.458837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.458837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.462691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.469150 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 351/708] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:57.473456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.588325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e4d9-4618-4f64-8c70-ba906e878c1b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.623184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.628770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.629814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.630099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.630366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.631127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:57.631433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.632861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.653091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 358/709] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:57.658565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.687564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754460a-9b70-4577-ba86-40dd37d57b03 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.744979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.745599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.746978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.747262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.747903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:57.748204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.749592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.757164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 352/710] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:57.758001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:57.823269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:57.864791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed338b7-f17e-462a-a3e5-499f0407abe9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.910166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6d5c5-44a6-4d6f-9dbf-19ade9169d0d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:57.941916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.942930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.943403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.943862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.963674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:57.964341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:57.964615 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:57.964885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] PATH_INFO: `/v3/services/network` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:57.965466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:57.967040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:57.979467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:57.980876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:57.981706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 353/711] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:57.987074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.009415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:58.031791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.033658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:58.104761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83bf4e-720c-4121-81b3-36f09cc52898 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:58.133689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:58.145310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:58.146200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.146656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.147145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.148069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] RBAC: Adding query filter params (name=network) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:58.148543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.150126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:58.156138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 354/712] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:58.163780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.285711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 359/713] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:36:58.290190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.371981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ce7c7-966c-4cc4-8fd2-b64ee32d0125 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:58.417563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:58.439043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b68e79-c805-4c16-a6d7-307b321abfbb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:58.443981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:58.444699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.445040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.445377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.446075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] RBAC: Adding query filter params (type=network) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:58.446445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.447938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbc457-41c1-4b01-816d-3d3a348be321 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:58.454181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 355/714] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:36:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:58.459862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515d2a0-5bf9-4bc6-bbb9-9f029d7c1611 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.460139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515d2a0-5bf9-4bc6-bbb9-9f029d7c1611 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.460405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515d2a0-5bf9-4bc6-bbb9-9f029d7c1611 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.461792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 356/715] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:58.468660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.468975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.469202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.506797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:58.508033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:58.510735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:58.511418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.511695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.511960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.531401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.532954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:58.556230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:58.557262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:58.558149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:58.558149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:58.558149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:58.566322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 360/716] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:58.567355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.626406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:58.651939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 357/717] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:58.713681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04101e5f-1556-4137-8daf-d56c890bf18b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:58.748966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:58.749582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.749859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.750132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.750773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:58.751161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.752532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef822b44-1096-485e-a580-b2e8101ebf15 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:58.754918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:58.766966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 361/718] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:58.772859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.819985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ba3be-d591-437d-b7d2-197f767ebcf9 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.820298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ba3be-d591-437d-b7d2-197f767ebcf9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.821795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ba3be-d591-437d-b7d2-197f767ebcf9 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.822251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 362/719] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:58.827832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.828120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.828398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.864407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:36:58.865622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:36:58.928469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dca329-6e45-4e2e-bb8e-61aef4681404 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:58.979715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:58.980870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:58.981151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:58.981418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:58.984930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 363/720] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:36:58.985876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:58.996357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:58.997908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:36:59.017293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:59.018232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:59.019519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 358/721] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:36:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:59.031825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.118710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68091d-e81c-40df-960c-afa3d832be3d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.128098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.146922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.159140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.160060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.161178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.161638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.183255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:59.191271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:59.192211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:36:59.193031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:59.197540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 364/722] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:59.204077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.245054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4b2c-afe4-4e3d-882d-35be2a2db4d2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.306877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea273d-0165-44d6-b9a7-4a241d8b8fb4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.320146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.320769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.321050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.321322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.322020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:59.322562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.323795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.336818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 359/723] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:59.337790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.345951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.347295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.347922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.348412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.349588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:59.350390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.352421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.364293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 365/724] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:59.365423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.429526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.456287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9bf54-9f66-41fe-a341-ce740fe37741 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.467325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.490190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6272-0b1c-402c-a804-b48a08dd7405 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.501785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.502360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.502907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.502907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.513491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:36:59.518429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:59.519371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:59.520196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 360/725] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:59.520887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.535817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.537369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.537846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.538531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.539696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:59.540198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.542159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.550614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 366/726] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:59.551901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.641175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084e40-f966-495e-a926-00d33513e71d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.684738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7924-637f-4ad0-bcb3-afe1d4086696 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.700589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.723167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.723919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.724275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.724572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.725762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.727207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.727685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.728139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.729009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:59.729485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.731390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.742176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 367/727] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:36:59.750330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.750737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.752069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:59.776545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:59.777555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:59.778618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:59.778618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:59.778618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:36:59.778618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 361/728] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:59.784886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.807572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:36:59.888152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de75b7-ba1d-4a7d-bb78-e3d353a6fcaa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.917804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf420d7-49f3-4ed4-ad18-ca30f8116d50 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:36:59.943550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.944192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.944470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.944742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.959855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:36:59.960042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.960680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.961055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.961424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.961524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.962232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:36:59.962631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:36:59.964151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5fb49-516d-47c4-8c6f-52af8ef8cc85 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:36:59.977482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 362/729] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:36:59.978158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:36:59.979346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:36:59.980992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:36:59.980992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:36:59.980992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:36:59.980992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 368/730] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:36:59.981285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aab2eb-0f79-4d1f-ad3a-6cb6d621fac3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.981285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aab2eb-0f79-4d1f-ad3a-6cb6d621fac3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.981285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aab2eb-0f79-4d1f-ad3a-6cb6d621fac3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:36:59.986879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 363/731] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:36:59.987914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:36:59.988107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:36:59.988396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:36:59.988668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.007636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:00.008768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:00.023711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.084192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 364/732] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:37:00.085056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.104731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.146990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6770ca-1e3f-4cb1-9bf6-585c06dc5fd4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.208512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.209155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.209433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.209704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.228189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.229643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:00.255652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:00.256663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:00.257582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:00.258170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 369/733] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:36:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:00.263192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.281087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cffe03-49d9-4261-9b84-3e3dfaee4858 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.323952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.326857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.327144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.327417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.328073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:00.328380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.329823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.358731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 365/734] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:00.380308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.434003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.454324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5a763-919b-4aad-9264-e94297f775e8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.474421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.491725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.492746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.493286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.493746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.494694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:00.495275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.497003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.516044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 370/735] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:00.517280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d551b-7dcd-43b0-93d9-c813f2ed75bf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.517492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.568260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.568938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.569223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.569499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.580029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.582568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:00.597850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:00.598786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:00.600005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:00.602901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 366/736] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:00.603802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.629900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec693-4301-4d4e-91cc-159d2db06ee8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.667769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.668380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.668806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.669081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.670113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.671805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.694664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 371/737] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:00 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:00.695655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.707292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.772076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feeab-a607-4885-a131-dae5b390a774 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.803924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:00.819430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.820218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.820613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.820986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.851795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d5a8-1cdf-4829-b87b-d6288853126d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:00.892550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:00.893162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.893439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.893704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.894557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:00.894997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.896411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954182-ee5c-4051-9f32-147cf0682f38 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.897512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:00.898995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8dd1c-7578-41f3-b34b-1ba26bfdb8bd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:00.916583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 372/738] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:00.917756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffcd4-8da3-4794-92e4-4c0b922a2406 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:00.918144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffcd4-8da3-4794-92e4-4c0b922a2406 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:00.918415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffcd4-8da3-4794-92e4-4c0b922a2406 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:00.919911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 373/739] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:00.923603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ffcd4-8da3-4794-92e4-4c0b922a2406 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:00.997121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.028202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 367/740] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 03:37:01.029394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.029674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.029942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.047641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ffcd4-8da3-4794-92e4-4c0b922a2406 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.048936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:01.050111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:01.068349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.092207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.093357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.093645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.093927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.135006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 368/741] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 03:37:01.136112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.298975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b173e9-1a01-4ab2-9429-345cc09a71ff None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.333828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.334517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.334797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.335121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.345506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.346998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.347925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.349579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:01.362065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:01.363085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:01.363974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:01.363974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:01.363974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:01.370240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 369/742] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:01.377147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.398829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.427335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.465934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 374/743] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 03:37:01.478174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.578670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca935f97-c9dc-4efe-9ba0-c56eb70b5457 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.587471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0accc-a7d7-4bbf-b301-3e55528f7e3e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.614386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.615185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.615557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.615920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.627534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.629353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.632236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.632842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.633117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.633387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.634033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:01.634328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.635710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c315405-ecea-4181-a4b5-3d2a241fb821 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:01.645150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:01.646085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:01.647426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:01.647426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:01.647426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:01.647832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 375/744] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:01.648788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.652264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 370/745] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:01.653604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846d1f-cf80-425f-8d55-6a89ef178aa3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.653894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846d1f-cf80-425f-8d55-6a89ef178aa3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.654169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846d1f-cf80-425f-8d55-6a89ef178aa3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.655561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 371/746] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:01.656401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57846d1f-cf80-425f-8d55-6a89ef178aa3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.669686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.728414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.762529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57846d1f-cf80-425f-8d55-6a89ef178aa3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.763136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa13a9-82c4-4777-aa70-13634cdb85fa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.794931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.795537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.795809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.796077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.796714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:01.797012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.798402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.801059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:01.801888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.802379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.802876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.803732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:01.805401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49ba1-04fe-4ee2-9608-6df40b81c920 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:01.809127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 372/747] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:01.809919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.825731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 376/748] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:01 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:01.827308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:01.827795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:01.828266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:01.862650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:01.862650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:01.936439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736975-18c5-49f9-b61c-858fb103e65c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:01.948115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 377/749] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:01.949001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:01.963211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:01.986401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:02.009933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.010686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.011130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.011407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.023659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.025309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:02.039714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:02.040635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:02.041525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:02.045785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 373/750] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:02.047125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.055438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f739d2e-b527-472a-99df-b9e15d8365d0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.100808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.101441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.101952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.102220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.111495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.113870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:02.128983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:02.129897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:02.130781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:02.136105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 378/751] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:02.140947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.170952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71582-cd75-4cb5-9bc0-5e6688ac9a56 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.201186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:02.217233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.218182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.219068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.219247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.219963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:02.220298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.221783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.242659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 374/752] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:02.243954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.252623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4a592-91df-4555-90df-1abe6477fae3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.297273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.298625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.299121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.299828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.300697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:02.301446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.302729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:02.305365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.326475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 379/753] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:02.327773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.373443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21074cce-f041-4c56-92da-d95a1fb88a2e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.417742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.418920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.419817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.420264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.421446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=323c45a05bd241458486b8919c93dd2e) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:02.422165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.424224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.432308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 375/754] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/role_assignments?user.id=323c45a05bd241458486b8919c93dd2e&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:02.433345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.454714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f2f8f-edf0-4863-865a-35f8fad5f097 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.496706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:02.513719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.514437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.514907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.515094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.515961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=156bd0c3f4b8489888de3ca7b7342e09) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:02.516313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.517862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.531025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 380/755] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/role_assignments?user.id=156bd0c3f4b8489888de3ca7b7342e09&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:02.560934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8372dc-be25-4023-aaa4-fcabb44f18b7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.574157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:02.602662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.604089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.604539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.604978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.617555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:02.623239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:02.624287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:02.625226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:02.628031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 376/756] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:02.633982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.745524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173abdf-d92a-4c83-a139-006bc92f9fda None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.780430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.781284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.781701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.782137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.782997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:02.783434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.784919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae13eef-6b6b-4ce2-9718-d7f1e5879b9f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.794046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 381/757] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:02.799640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:02.900707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aea64-c705-4bda-adf4-adbe27e339a8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:02.929240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee351e9-d79d-4805-ba60-3065d97de4ec None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.929767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee351e9-d79d-4805-ba60-3065d97de4ec None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.929767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee351e9-d79d-4805-ba60-3065d97de4ec None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.931409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 382/758] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:02.935694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:02.936531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.936644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:02.936824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.937105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.937338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:02.937740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:02.950745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:02.952753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:02.956001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:02.957220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:02.970884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:02.972030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:02.973161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:02.973161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:02.973161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:02.975681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 377/759] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:02.983758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.022530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:03.034507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:03.070678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 383/760] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:03.152289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a304f8-b501-48da-9f69-d69d32f8dd6a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:03.200427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:03.201059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:03.201333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:03.201604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:03.202261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:03.202562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:03.204024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:03.220690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 378/761] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:03.222101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.345029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c1de8-11be-469e-a300-3389fbf0acfe None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:03.391310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:03.391976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:03.392257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:03.392524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:03.399676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:03.401180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:03.418485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:03.419435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:03.420347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:03.420862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 384/762] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:03.425639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.527626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:03.547520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5c4b8-9bac-4d7c-9168-01821d734077 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:03.560960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.606654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:03.607490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:03.607629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:03.607629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:03.607890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:03.608608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:03.609033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:03.610826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:03.649544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 379/763] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:03.653505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.733360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569c53-3014-42ef-8509-56a6fc4fe86f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:03.800980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:03.802791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:03.803090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:03.803365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:03.803961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:03.805336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:03.814914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98c8-a028-4ca1-beed-797885ea64c2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:03.825087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:03.826076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:03.826922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 385/764] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:03.838348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:03.884351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:03.885470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:03.885745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:03.886008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:03.905546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:03.911474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:03.913050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:03.940025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:03.941119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:03.941978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:03.941978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:03.941978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:03.941978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 380/765] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:03.950585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.032485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:04.035716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e9bda-8b7a-4cd5-b860-ae1dfc38176d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.086523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.087158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.087434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.087703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.089388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:04.089693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.091074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.100000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 386/766] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:04.107102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.153507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae145-1d3e-4c78-b59e-3a9f4e797e9f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.218060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.218675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.218972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.219240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.219929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:04.220224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.221610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.250841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 381/767] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:04.256516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.283550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871708b7-ad8d-4e73-9f6b-340bbbf9da11 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.324146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:04.328677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.330150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.330434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.330705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.331371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:04.331675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.333051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abb2ab-700d-41fe-a57f-321d32cc1e24 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.341594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 387/768] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:04.343150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f645-2330-4acd-8958-1e76140e9da4 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.343442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f645-2330-4acd-8958-1e76140e9da4 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.343722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f645-2330-4acd-8958-1e76140e9da4 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.346882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 388/769] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:04.347666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f645-2330-4acd-8958-1e76140e9da4 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.368746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b1f2-61d5-4a14-932c-c09f09415391 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.386527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:04.404415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.405100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.405375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.405644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.412989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.414568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f95a0ee3-3d35-4b71-b868-385f4ddc021d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:04.426476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:04.427569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:04.428734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:04.432547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 382/770] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:04.433787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.434065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.434336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.453291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:04.454659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:04.477451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f645-2330-4acd-8958-1e76140e9da4 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.516957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.518045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.518552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.519064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.519778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] RBAC: Adding query filter params (interface=public, service_id=292872575593458a9833874330e440c5, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:04.520130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.521618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.534852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 389/771] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/endpoints?service_id=292872575593458a9833874330e440c5&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:04.535877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.543008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 383/772] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:04.543008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.616853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:04.707425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5af0a-a4ee-4c17-ba05-e7bb18a3d72a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.710824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:04.726331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf9c6-c157-4666-bbaf-25be53b4b74d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:04.754200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.755305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.755869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.756424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.757228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.758847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e10ef4-324f-4227-a320-785f82980ac0 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.760776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:04.761527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.761893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.762250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.763058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:04.763452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:04.765059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9cdcf-32af-4833-a41f-b2e121d7cb31 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:04.766006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 384/773] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:04.767608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098eab0-dee0-4e0b-a531-d470b9964efb None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.768100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098eab0-dee0-4e0b-a531-d470b9964efb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.768565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098eab0-dee0-4e0b-a531-d470b9964efb None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.774276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 385/774] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:04.776729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.777217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.777685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.789488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 390/775] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:04.790748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d897ab-bc28-494f-a420-85cca36069a9 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.791073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d897ab-bc28-494f-a420-85cca36069a9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.791343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d897ab-bc28-494f-a420-85cca36069a9 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.793087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 391/776] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:04.798517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:04.798797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:04.799089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:04.800385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:04.801945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:04.818469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:04.819783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:04.890846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 386/777] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:04.890846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.906742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 392/778] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:04.908375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:04.942227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.003990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.024389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57acca0-5454-47db-8098-fb0f1148261b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.034966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e194631-45b6-4a32-89db-e276fb0d22c6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.066073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.066712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.066914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.067177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] PATH_INFO: `/v3/services/network` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.067759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.069127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.075806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.076485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.076766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.077039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:05.078089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:05.079135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:05.080469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 393/779] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:04 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:05.083623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=8e0e6be0fc61491fa25193834b4fa8af, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.085318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.085716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.102831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 387/780] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:04 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:05.181387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.210833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13249afb-b2e5-418b-b181-e0c2bbff457c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.258281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.259191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.259715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.260189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.261028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] RBAC: Adding query filter params (name=network) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:05.261533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.263722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.271319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 394/781] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:05.272725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.285885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.290653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.293465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785762bb-dd34-4705-8962-fcba026a37b7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.341316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.341637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.341912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.342180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.355430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:05.360197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:05.361358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:05.362410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 388/782] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:05.363147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.426925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c3ca-5551-4536-aa02-4252440cd347 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.462766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.463635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.463949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.464318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.467030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba4b4-2bf1-486d-a975-16af5fa35667 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.475381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:05.481097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:05.482266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:05.483391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:05.484727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 395/783] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:05.485840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.507036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.507938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.508403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.508821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.509607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:05.510091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.511707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.517431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 389/784] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:05.518593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.551171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.558658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.590128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa243-4ffe-4448-86a2-c520c2a2e890 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.625642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.626343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.626691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.627059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.627813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:05.628079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0ae65-fd96-4de5-877b-d650cb2e94a4 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.628511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.629608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.642760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 396/785] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:05.642760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.669929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.670884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.671440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.671913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.673190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:05.673501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.674918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.683984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 390/786] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:05.684906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.750592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c91b-498d-400d-82d9-f96b88fe16cc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.780970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.782782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:05.788799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.789595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.790021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.790438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.791484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] RBAC: Adding query filter params (interface=public, service_id=f4a7ef53cf9b4aa6b1c741ba749ef0f7, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:05.791939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.793475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.799320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 397/787] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/endpoints?service_id=f4a7ef53cf9b4aa6b1c741ba749ef0f7&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:05.800252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.805039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafac3cb-3c83-4f05-83ad-6ec37d583fe6 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.846401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.847309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.847656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.847984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.861789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.863552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:05.879219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:05.880167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:05.881043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:05.881043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:05.881043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:05.887766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 391/788] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:05.888861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:05.913021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965de2-32b5-4af5-be8d-8a1c8351e0af None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:05.961243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:05.961879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:05.962200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:05.962511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:05.973745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:05.975229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:05.990854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:05.991812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:05.992698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:05.992698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:05.992698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:05.992698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 398/789] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:05.992920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.000268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436253c-f3f8-48c1-9c02-9ff3617e8c4e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.043544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.044160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.044441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.044716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.045329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.046721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.047809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.049291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.062830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 392/790] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:06.062830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.151773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb0e4-0052-40c2-8257-8d802b7aee32 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.198441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3199fc-1eee-4ded-ac89-d122fd9ff7f5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.216829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.217901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.218372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.218879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.219922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:06.220415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.221988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.244718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.244718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.244718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.244718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.245148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:06.245148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.245148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.246855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 399/791] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:06.252829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.274771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 393/792] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:06.278636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.344242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.389839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fc90-b1ec-48c1-9aa1-f9a2e478fda2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.390496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc11bd-e76c-403b-b5eb-946f3743cddb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.400841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.426866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.427558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.427840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.428112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.428920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.429551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.429824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.430097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.436769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.437654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.438668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.439149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:06.450884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:06.452052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:06.453260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 400/793] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:06.454242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:06.455206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:06.456134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:06.456801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 394/794] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:06.457753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.463331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.575159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad8f5-4f53-49b6-85da-d4578eaa1254 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.612158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b8220-0408-43c7-9af0-a97e3846e1f7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.630026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.631665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.631945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.632212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.632947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:06.633248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.634679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.646483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.671169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 401/795] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:06.675034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.675651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.675921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.676185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.677759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.678256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:06.680836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.682283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.715883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 395/796] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:06.721808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:06.722853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:06.849231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b7ca-f9cd-4882-990e-eaba8278b174 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.903198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb4d48-d071-4179-8869-9d1015edc09b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:06.939712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.940350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.940679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.940955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.951178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.952641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.959526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:06.960179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:06.960458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:06.960734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:06.972368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:06.973408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:06.974294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:06.974294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:06.974294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:06.974294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 396/797] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:06.978580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:06.980366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:06.981647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a9059980-f4ae-40a7-b816-966e1415156c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:07.018270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:07.019307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:07.020191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:07.020191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:07.020191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:07.022565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 402/798] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:07.023992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c39a-34cf-4baa-8228-4d58c9f75963 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.024268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c39a-34cf-4baa-8228-4d58c9f75963 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.024694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c39a-34cf-4baa-8228-4d58c9f75963 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.026786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 403/799] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:07.027697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715c39a-34cf-4baa-8228-4d58c9f75963 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.038885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.114220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d06d-9999-435d-ba6a-0ed14eac2112 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.123356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.151082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715c39a-34cf-4baa-8228-4d58c9f75963 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.152711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.153588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.154032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.154463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.155279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:07.155741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.157347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1dcbf-a887-47d3-a0cb-22d87ba33335 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.171509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 397/800] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:07.173137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.173646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.174116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.188511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.189151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.189515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.189893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.190594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:07.191115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.192474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.195457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:07.196884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:07.205595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 404/801] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:07.206426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.271084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 398/802] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:07.272116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.296826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.311230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80310-f7af-472a-97f6-f144f60e22de None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.348664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.349683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.350208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.350484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.360856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.362291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.364663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-3a3f0593-1fc2-46e5-8c38-9f9ae8543d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:07.379672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:07.380627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:07.381523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:07.382579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 405/803] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:07.383914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd36de-98ad-4bd2-be8c-adc8233fc86d None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.384243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd36de-98ad-4bd2-be8c-adc8233fc86d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.384515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd36de-98ad-4bd2-be8c-adc8233fc86d None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.386034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 406/804] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:07.388734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd36de-98ad-4bd2-be8c-adc8233fc86d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.412144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaaf13-5da8-4653-aa0e-6a38bcf8c9bd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.453167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.454210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.454685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.455195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.462351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.464076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a5f6df52-f75b-4afd-a7ce-358db292a2a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:07.475743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:07.476653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:07.477607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:07.480332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 399/805] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:07.481749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.482224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.482702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.500985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:07.502891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:07.510019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd36de-98ad-4bd2-be8c-adc8233fc86d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.547677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.548390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.548666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.548937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.549636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:07.549934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.551401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.563139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 407/806] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:07.566901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.577951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 400/807] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:07.579124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.589668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.641752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.699338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645ea6-ce34-4bbb-9c1e-19239381f088 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.704324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a2d-bbea-4c33-b332-c0b26235d970 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.736196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.736828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.737105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.737371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.737953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.739376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:07.749338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:07.750379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:07.756419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 401/808] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:07.757191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.757266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.757266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.757266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.757266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.757545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:07.757545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.758983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.772295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 408/809] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:07.773198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.851559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.878874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d05cf3-8db9-4686-bc90-2f188c061943 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.886040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b19-32c5-4dcf-a408-5ecb5f336212 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:07.898226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:07.915932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.916546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.916824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.917101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.917721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:07.918021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.919428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.921223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:07.921750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:07.922025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:07.922294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/323c45a05bd241458486b8919c93dd2e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:07.924598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 409/810] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:07.925264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:07.929258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=323c45a05bd241458486b8919c93dd2e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:07.930382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:07.944786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 402/811] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:07 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/323c45a05bd241458486b8919c93dd2e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:07.945602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.054553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b139d-6a45-48e5-af01-5203ffa74437 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.095096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.095801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.096078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.096349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/156bd0c3f4b8489888de3ca7b7342e09/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.096955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01053-a622-468b-82d8-4afd18076766 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.113013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=156bd0c3f4b8489888de3ca7b7342e09, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.114720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.129074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 410/812] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:07 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/156bd0c3f4b8489888de3ca7b7342e09/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:08.129867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.134367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:08.144305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.146540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.147050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.147458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.159971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:08.165740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:08.166708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:08.167639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:08.168940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:08.170121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 403/813] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:08.177081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.282181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabdb8-b1a8-4f40-bff2-2313a2c13ffa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.348948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.349547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.349828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.350436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.351689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:08.351689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.352876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68c1f-e16c-4733-bd02-37e4f86d0346 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.359438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 411/814] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:08.360951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5ce6-40df-4933-871e-d7c6055ae6a2 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.361397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5ce6-40df-4933-871e-d7c6055ae6a2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.361711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5ce6-40df-4933-871e-d7c6055ae6a2 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.363449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 412/815] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:08.368075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e5ce6-40df-4933-871e-d7c6055ae6a2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.382387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64dc88-b80e-402d-b40b-a6980c8a8af7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.420558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.421440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.421717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.421986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.422961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:08.423709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.425358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda3e6-cc6b-42b1-a59c-44f7a21822d6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.436714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 404/816] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:08.437939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.438287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.438559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.461036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:08.462504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:08.478669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:08.486868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e5ce6-40df-4933-871e-d7c6055ae6a2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.526782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.527366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:08.527499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.527669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.527936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.528600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.530029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.548444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 405/817] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:08.549660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.566450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 413/818] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:08.664677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0fc52-5450-48a0-9b9c-6743e5da5018 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.707001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.708511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.708985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.709449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.720787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.722636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:08.739175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:08.740198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:08.741098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:08.741098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:08.741098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:08.746407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 406/819] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:08.749547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.858233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e936-694d-47c7-a3ef-d53283a0f127 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:08.893662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:08.894666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:08.895274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:08.895798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:08.896632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:08.897125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:08.898682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:08.910758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.912030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 414/820] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:08.919024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:08.937965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:08.959209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.028455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cbd5-e923-4063-a47d-d69c9c7e1f7c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.036245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e37f64-6a36-415f-86be-d898bcffeafd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.070907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.071638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.071921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.072191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] PATH_INFO: `/v3/services/network` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.072780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.074161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.076561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.077510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.078058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.078332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:09.086654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:09.087709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:09.088643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 407/821] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:09.089426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.090130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.097139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:09.114867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:09.115955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:09.117006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 415/822] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:37:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:09.123015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.202583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.208769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1c0d-e444-4f0c-bec9-54ed00d81767 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.219269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.250220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.250220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.250220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.250220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.250695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] RBAC: Adding query filter params (name=network) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:09.250695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.254965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.259198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 408/823] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:09.265920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be415b82-f791-4c2e-a7bc-eb3947461fd7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.274582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.305427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.306626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.307099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.307517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.308372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:09.308802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.310371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.328590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 416/824] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:37:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:09.336186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.440991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78569-1a93-472d-8704-272e963e96a6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.441435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdc013-fdf5-43b1-a892-d0a37d2eddb2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.472931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.481984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.484408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.484695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.485129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.509578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.510199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.510476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.510747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.511415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] RBAC: Adding query filter params (type=network) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:09.511716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.514272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.524282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 409/825] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:09.533483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.582510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.629630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.631434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.694459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa90fa-00a5-4723-b68f-5f38dbfbf033 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.729002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.729613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.729841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.730379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:09.731137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:09.732558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:09.755528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 410/826] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:09 2022] POST /identity/v3/endpoints => generated 337 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:09.787434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 417/827] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:09.794826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:09.905088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:09.948586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98adf3-f336-407c-83d9-b6d782f15c9b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:09.996208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:09.996785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:09.997062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:09.997335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.009076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.010555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-64c45202-2189-4ff6-8f1d-59710967e032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:10.025635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:10.026611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:10.027542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:10.027542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:10.027542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:10.027542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 411/828] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:10.032485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.088505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.155095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280172a1-e2b7-463a-842d-b85e7b4c49b0 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.155410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280172a1-e2b7-463a-842d-b85e7b4c49b0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.155644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280172a1-e2b7-463a-842d-b85e7b4c49b0 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.157332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 412/829] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:10.162657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.162972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.163246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.167710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8eee9-0b02-408d-94c7-87d3ae0712c4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.183672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:10.184862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:10.204035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.204647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.204926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.205198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.205838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:10.206140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.207646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.219396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 418/830] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:10.224783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.271813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 413/831] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:10.326279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.335659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5267-fdb6-46b0-9056-1b0ed6385d5e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.369601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.370225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.370497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.370765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.377417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.378932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-d659739b-52d3-45a7-92b7-fdbe77a595d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:10.389965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:10.390874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:10.391860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 419/832] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:10.398531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.428352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68f7-052d-4a43-9cdd-1dabbac817aa None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.428962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68f7-052d-4a43-9cdd-1dabbac817aa None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.429429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68f7-052d-4a43-9cdd-1dabbac817aa None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.431276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 420/833] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:10.436261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.436760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.437231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.456750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:10.458541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:10.461943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.517220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba6a01-a0f1-45de-971d-e64e3c9db289 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.534412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 421/834] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:10.535426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.559540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.560456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.560742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.561250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.561960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:10.562407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.564496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.579165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 414/835] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:10.584899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.633970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.658493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76748d26-e7c3-493f-bd2a-94dd917b8cbf None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.699863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.700517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.700795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.701068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.712551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:10.717673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:10.718601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:10.719647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 422/836] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:10.724641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.813337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e34a97-3929-4b17-849c-fb81eca5d9c5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.831587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.833759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7713f-7fd3-4548-b5ad-348bc840cc9e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:10.850389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.851165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.851379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.851643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.852468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:10.852768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.854197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.861412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 415/837] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:10.871241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.880099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:10.880848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:10.881227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:10.881561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:10.882324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:10.882685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:10.884251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:10.894278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 423/838] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:10.899697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:10.903835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:10.994567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f35f2-afbc-48ff-90fd-8b383a8c1f81 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.034434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.035235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.035511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.035936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.047409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eddbf-d02e-4a6e-bff0-07f41d38abbe None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.049384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:11.054408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.055378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:11.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 416/839] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:11.063079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.098305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.099066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.099346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.099855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.114451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.115792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.116609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:11.134747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.135869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:11.136767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:11.136767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:11.136767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:11.136767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 424/840] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:11.142208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.187916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03442a1c-fd14-49bd-8e8f-14c4ade874c8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.197701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.221717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.222335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.222619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.222913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.223586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:11.223890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.225280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c347fe5-062d-4d57-b73f-96069ea31ed9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.236264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 417/841] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:11.237492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d6f5-b7d5-4f51-bdf8-c88fb2f96d03 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.237765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d6f5-b7d5-4f51-bdf8-c88fb2f96d03 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.238037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d6f5-b7d5-4f51-bdf8-c88fb2f96d03 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.242854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 418/842] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:11.242854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d6f5-b7d5-4f51-bdf8-c88fb2f96d03 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.267658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f897d7-99ff-40c0-ad0a-1480c1e117ba None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.303542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.304313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.304590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.304898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.305611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:11.306146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.307636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f01517-4e79-4838-abd3-917eb7d36975 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.320549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 425/843] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:11.321475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.321760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.322034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.340333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:11.341532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:11.356280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d6f5-b7d5-4f51-bdf8-c88fb2f96d03 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.359319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.394898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.395535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.395811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.396080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.406571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.408132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.422986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 426/844] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:11.422986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:11.423245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.424247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:11.425120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:11.425120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:11.425120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:11.425120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 419/845] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:11.429759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.482902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.538143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c2340-1b18-4095-916e-9f56f74c7dfa None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.556078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555219a-bde7-4489-bce7-180e8946487c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.572890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.573871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.574373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.574876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.582328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.584067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:11.596969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.597926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 03:37:11.598895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:11.599425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.599425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.599577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.600173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.600466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:11.600769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.602238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.602455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 427/846] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:11.610554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.616676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 420/847] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:11.621974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.649258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.722255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac3b73-001d-443c-a4dc-3bd8cb5404dc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.724520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.740657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c075-cd25-411b-8500-9906cd288156 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.756045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.756654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.756931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.757200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.757922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:11.758225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.759693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.776560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 428/848] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:11.778253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.779157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.779612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.780059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.781968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.787898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.789370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:11.801535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.802507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 421/849] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:11.803497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.887577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:11.892596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf5d8-4733-4e8c-b893-5862460fc32a None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.912039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165276-0c5d-45b3-b805-225b39191574 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:11.929319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.929933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.930218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.930469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.931073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:11.932468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:11.941097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:11.942117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:11.943005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 429/850] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:11.943802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:11.954914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:11.955531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:11.955805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:11.956073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:11.979078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.028244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.029921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.054775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27fd0e-2000-4f3b-82df-a338181888f2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.089824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.090436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.090715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.091027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.091748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.092053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.093477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.110974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.116265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 430/851] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:12.116265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.139246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 422/852] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:12.140173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.203962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.221090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506804a7-b746-490e-8f4e-9a5019d27eb7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.243506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036b489-7391-4676-8740-c7669ac801ca None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.262312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.262949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.263226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.263492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.264112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.264407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.265770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.274797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 431/853] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:12.274797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.286304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.286958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.287232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.287499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.332110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.355130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.357097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.386643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642d2b2-b1a7-49c8-a8f5-605fa3376765 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.421892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.422535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.422848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.423104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.434690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.436197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.444700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:12.452253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:12.453201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:12.454200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:12.454200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:12.454200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:12.454200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 432/854] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:12.454200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.464177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 423/855] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:12.465117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.548746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.568513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77484707-677c-44ff-b51c-70c8cf0d7d50 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.572246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326a30c-aae7-4920-91b2-719546f4d8b2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.603093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.603694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.603971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.604237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.605171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.605468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.606606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.606839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.607239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.607512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.607769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.608405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.608706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.610161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.623856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 433/856] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:12.623856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.634985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 424/857] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:12.640563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.702958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.749325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e18020-892d-423f-a03d-9c54b5fe555d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.766407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41d9fc-caf6-462d-9a53-d7949a071989 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.792136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.792812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.793071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.793341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.804671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.806114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.809579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.810224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.810534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.810768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.813875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.818305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.820048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:12.822378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:12.823386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:12.824273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:12.824273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:12.824273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:12.824273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 434/858] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:12.832663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:12.837770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:12.838781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:12.839687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 425/859] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:12.844934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:12.943376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37126754-d1f5-48cc-8150-5347381525a5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.957152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53506bd-ee38-4869-9539-27ba8d571896 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:12.963516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:12.987393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.988007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.988304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.988555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.989203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.989523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:12.990872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772da1dc-029b-49fd-bb54-53d252d0edc9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:12.991274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:12.991548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:12.991813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:12.992491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:12.992788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:12.994197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.008991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 435/860] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:13.009177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 426/861] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:13.009290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4391-57f4-42fd-b4ac-509e4d937ae7 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.009290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4391-57f4-42fd-b4ac-509e4d937ae7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.009290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4391-57f4-42fd-b4ac-509e4d937ae7 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.009290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 436/862] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:13.010832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.011952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4391-57f4-42fd-b4ac-509e4d937ae7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.073506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:13.119105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f5957-759e-4ed9-880b-5dca75ff27ac None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.125738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4391-57f4-42fd-b4ac-509e4d937ae7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.152979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.153616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.153897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.154168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.161032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.162486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.162667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.163287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.163647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.163921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.165428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=323c45a05bd241458486b8919c93dd2e) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:13.165920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.167383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17a554-af39-4b1b-b0d3-f399080e2a8a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.180474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 437/863] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/role_assignments?user.id=323c45a05bd241458486b8919c93dd2e&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:13.180665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:13.181504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:13.182387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:13.182847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.182847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.182847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.183069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 427/864] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:13.183069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.201707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:13.203275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:13.257731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:13.299285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 438/865] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:13.326172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11827-f5d9-4364-93d9-533bf88764d5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.343316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:13.363417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.364114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.364458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.364791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.365551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:13.365918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.367473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.380703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 428/866] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:13.386768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.491271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173c46-bee1-4a3b-8adb-f69e4a50752a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.525742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.526584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.527083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.527491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.528398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=156bd0c3f4b8489888de3ca7b7342e09) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:13.528905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.530551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.540900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 439/867] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/role_assignments?user.id=156bd0c3f4b8489888de3ca7b7342e09&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:13.569948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.688948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f162646-0e69-4d07-b177-618475f13308 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.727138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.727779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.728055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.728328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.728915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.730589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:13.739742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:13.740714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:13.741820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 429/868] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:13.749541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.786952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:13.860536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965563df-4128-42e2-8c9f-434cfb17e629 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:13.895844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:13.896461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:13.896743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:13.897016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:13.897649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:13.897953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:13.899400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d1d3-faef-4567-b4e6-8b2cf701dde8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:13.904305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 440/869] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:13.910021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:13.953145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:14.011756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a2c21-6ed6-4b28-a6f4-daffaa620f37 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:14.056865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:14.057402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:14.057680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:14.057952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:14.058585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:14.058908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:14.060356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06985c-eba6-46f3-af46-8ec42eb98975 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:14.065289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 430/870] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:14.572800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6c94-81a0-4a3f-9daf-1d2ed62724c1 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:14.573403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6c94-81a0-4a3f-9daf-1d2ed62724c1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:14.573403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6c94-81a0-4a3f-9daf-1d2ed62724c1 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:14.574948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 441/871] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:14.580118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5df2-778a-428b-892e-23fe3be86eb4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:14.580400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5df2-778a-428b-892e-23fe3be86eb4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:14.580670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5df2-778a-428b-892e-23fe3be86eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:14.598698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9c3f5df2-778a-428b-892e-23fe3be86eb4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:14.599902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9c3f5df2-778a-428b-892e-23fe3be86eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:14.670936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 431/872] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:14.928570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea6c94-81a0-4a3f-9daf-1d2ed62724c1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:15.026102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:15.031301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea6c94-81a0-4a3f-9daf-1d2ed62724c1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:15.067726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:15.068342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.068617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.068888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.069559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:15.070954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3876c9-2c39-4430-992a-871f53d0474a None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:15.086783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 442/873] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:14 2022] POST /identity/v3/services => generated 251 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:15.209703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aaefe-26e8-446b-9805-4b7a3a5a1137 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.210698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aaefe-26e8-446b-9805-4b7a3a5a1137 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.211222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aaefe-26e8-446b-9805-4b7a3a5a1137 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.213147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 432/874] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:15.219200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29fb33-086d-408f-baaa-cc10c239727b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.219512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29fb33-086d-408f-baaa-cc10c239727b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.219813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29fb33-086d-408f-baaa-cc10c239727b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.245848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1c29fb33-086d-408f-baaa-cc10c239727b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:15.247132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1c29fb33-086d-408f-baaa-cc10c239727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:15.325805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 443/875] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:15.593174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aaefe-26e8-446b-9805-4b7a3a5a1137 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:15.631690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:15.683819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa635-fd60-4264-aeb7-257f59dbaef6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.684339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa635-fd60-4264-aeb7-257f59dbaef6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.684339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa635-fd60-4264-aeb7-257f59dbaef6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.685997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 444/876] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:15.691860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10743-e365-421f-9de1-69afc0589310 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.692310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10743-e365-421f-9de1-69afc0589310 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.692696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10743-e365-421f-9de1-69afc0589310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.703316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aaefe-26e8-446b-9805-4b7a3a5a1137 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:15.716260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-45c10743-e365-421f-9de1-69afc0589310 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:15.717945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-45c10743-e365-421f-9de1-69afc0589310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:15.739152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:15.739784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.740056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.740325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.741123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:15.742372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:15.751839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:15.752822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:15.753655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 433/877] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:15.761329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:15.815378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 445/878] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:15.860104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:15.863292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159dca-6678-41ac-a203-0a536f4235ba None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:15.899351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:15.899965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:15.900242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:15.900510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:15.901141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:15.901439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:15.902827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2770-feaa-43c3-84d9-76a9ca29a980 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:15.907933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 434/879] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:15.913582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c10743-e365-421f-9de1-69afc0589310 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:15.935134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:16.004745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9756-d50b-46a9-a77d-6565f36b7a12 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.005073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9756-d50b-46a9-a77d-6565f36b7a12 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.005305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9756-d50b-46a9-a77d-6565f36b7a12 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.006945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 435/880] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:16.011803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.012085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.012362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.019085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c10743-e365-421f-9de1-69afc0589310 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.032944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:16.034140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:16.059368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.060465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.060966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.061723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.062883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:16.063600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.065849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.071995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 446/881] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:16.105927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.109876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 436/882] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:16.201040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:16.224544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3d193-284f-464f-ada0-b24177480500 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.269007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.269894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.270357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.270772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.283259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:16.289173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:16.290364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:16.291391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:16.293577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 447/883] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:16.299368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.375856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.395582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:16.405981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f58324-710d-4a39-9e53-dd9baa6c7455 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.441081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.441696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.441974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.442245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.442946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:16.443251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.444696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.453495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 437/884] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:16.458617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.514335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb474101-bb04-4a8f-ad82-e03bd4ce3177 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.555671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.556588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.556886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c831-54bc-423d-9f45-6ea374c9945c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.557244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.557684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.558533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.560113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.575217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:16.593891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 448/885] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:16 2022] POST /identity/v3/services => generated 269 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:37:16.609158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.610296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.610918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.611400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.624065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.625977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:16.642947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:16.643993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:16.644922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:16.644922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:16.644922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:16.652043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 438/886] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:16.653816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.766353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70dd-ee36-4839-85d5-c241de7182ff None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.800747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.801359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.801636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.801901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.802539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:16.802928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.804220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.817736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 449/887] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:16.823615 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:16.826692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:16.931014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220daecc-a8d9-4616-8ddd-3a59ed0f88ad None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:16.964511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:16.965198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:16.965471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:16.965737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:16.973284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:16.974778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-4f26d9f6-6df2-47c6-9b3b-12e295eb47a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:16.986715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:16.987711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:16.988890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 439/888] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:16.995771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.055933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.098358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8064e-c747-47f9-89ca-998ae80b70fb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.133454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.134064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.134337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.134440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b622e-3ac4-49f3-ad46-b68ea8279859 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.134617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.134713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b622e-3ac4-49f3-ad46-b68ea8279859 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.134912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b622e-3ac4-49f3-ad46-b68ea8279859 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.135371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:17.135670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.136511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 440/889] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:17.137157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.141614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.141899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.142177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.159443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 450/890] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:17.163808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:17.164983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:17.165216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.226776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.243121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 441/891] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:17.290125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f3d06-3fb2-4b0e-86db-1a49b3003f0e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.306964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.325678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.326282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.326554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.326835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.393501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.394966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.500683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 451/892] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:37:17.502101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.520216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.541990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.621379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503cf22-b55b-4cf2-8c99-13c46bb124c1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.644115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42351951-3ff7-4311-9b8d-e33f09517671 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.663137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.663916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.664558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.664835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.677644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.679593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.687891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.688804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.689679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.690169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.691220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.692777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:17.695988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:17.696924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:17.697798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:17.697798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:17.697798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:17.701945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 442/893] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:17.703218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:17.709139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:17.710142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:17.711008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:17.719448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 452/894] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:17.721416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.766748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.814123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.824215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0635c38-2728-4fb4-8f70-4031099ecc9d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.835484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4dac-d5e3-48d5-94b1-23fcc7f8ddf0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.862869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.863495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.863778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.864052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.864701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:17.865007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.866570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.874373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:17.875328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:17.875809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:17.876281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:17.877102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:17.877605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:17.879230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:17.883652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 443/895] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:17.886913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.889343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 453/896] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:17.890282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:17.989614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89101fd-7350-4b50-a10a-303573f5769a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:17.992300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:17.993001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a95f4-b308-4a72-bb3e-acb77688145b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.027440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.028053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.028533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.028872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.029172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.029315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.029686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.029839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.030579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:18.031090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.032728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.037747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.039465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.043146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 454/897] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:18.044101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7a385ae7-ff8f-4d38-9d48-3d449790f983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:18.055824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:18.056941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:18.058089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:18.058731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:18.060083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 444/898] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:18.061592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efee6-02b7-492d-9458-4289de4599b9 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.062066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efee6-02b7-492d-9458-4289de4599b9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.062530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efee6-02b7-492d-9458-4289de4599b9 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.064281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 445/899] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:18.069545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efee6-02b7-492d-9458-4289de4599b9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:18.149937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f400f-e7f8-4e47-a604-e6cab5d9c598 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.169588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efee6-02b7-492d-9458-4289de4599b9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.184160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.184806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.185085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.185356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.186049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] RBAC: Adding query filter params (interface=public, service_id=9e69b15f709b47889bd17a3256050709, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:18.186358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.187921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb22ae-af2c-418b-b7e2-c28a2f317b6d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.193499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 455/900] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:18 2022] GET /identity/v3/endpoints?service_id=9e69b15f709b47889bd17a3256050709&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:18.197814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.197814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.197814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.207510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.208114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.208390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.208659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.209410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:18.209823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.211287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.215202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:18.216390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:18.223467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 446/901] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:18.228888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:18.258485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:18.295185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 456/902] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:18.296379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:18.298679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:18.347798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da0fb-7845-4efc-8bec-49f2729ca244 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.384171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.385015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.385424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.385950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.386784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.388428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.394934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 447/903] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:18 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:37:18.412459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162e5e1-f068-4469-b4eb-190b76522a8c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.451162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.452088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.452509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.452916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.453883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] RBAC: Adding query filter params (role.id=5bec5ed56d5a4a44978a688ccd60fb61, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=156bd0c3f4b8489888de3ca7b7342e09) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:18.454363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.456229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f2a9e-73f8-487b-9609-7077aa2a45fe None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.463299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 457/904] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:18 2022] GET /identity/v3/role_assignments?user.id=156bd0c3f4b8489888de3ca7b7342e09&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=5bec5ed56d5a4a44978a688ccd60fb61 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:18.572492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:18.667518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a8-aba7-4032-90be-677197ca0b10 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.667726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a8-aba7-4032-90be-677197ca0b10 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.667972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a8-aba7-4032-90be-677197ca0b10 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.669562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 458/905] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:18.671391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:18.674992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.675277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.675678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.684970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257b80-52c1-4f17-9364-11d43f3f039f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:18.695235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:18.696725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:18.720769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:18.721432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:18.721703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:18.722003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:18.722672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:18.724113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:18.744971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 448/906] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:18 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:18.775469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:18.792898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 459/907] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:19.056657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:19.162192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04283d-f7d5-41af-9c1d-af69d3401767 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:19.199385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:19.200324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:19.200806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:19.201246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:19.202001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:19.203704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:19.212859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:19.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:19.214676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:19.217013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 449/908] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Sep 3 03:37:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:19.222876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:19.324577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423bd3f-8069-4d1e-92ad-c83cc508c720 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:19.359856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:19.360467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:19.360746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:19.361014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:19.361637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:19.361935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:19.363375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:19.368566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 460/909] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:19.375929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:19.391903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:19.486947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5196afd-0219-438b-b34d-16223c2adbcb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:19.521069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:19.521678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:19.521952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:19.522221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:19.522869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:19.523173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:19.524552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:19.529811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 450/910] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:19.535476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:19.596871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:19.644112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7e48-76d4-44e7-94a4-a19cc273e682 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:19.678847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:19.679463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:19.679744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:19.680015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:19.680703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] RBAC: Adding query filter params (interface=public, service_id=8eff48f1955840dea5f2523dff79c6c6, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:19.681010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:19.682374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b85a3-d6c2-46f1-b723-d15ec7c2624b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:19.687845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 461/911] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:19 2022] GET /identity/v3/endpoints?service_id=8eff48f1955840dea5f2523dff79c6c6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:19.693298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:19.772286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:19.795784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d0172-9a98-42d5-978c-5aaf4458a3cf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:19.830642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:19.831289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:19.831589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:19.831829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:19.832436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:19.833807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673432-e84a-4c83-a60a-d4bcf48f3e4b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:19.840421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 451/912] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:19 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:37:20.421096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e40a8-096e-4dd9-b47b-19e65ff274bd None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.421619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e40a8-096e-4dd9-b47b-19e65ff274bd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.421619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e40a8-096e-4dd9-b47b-19e65ff274bd None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.423267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 462/913] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:20.428709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08b7fe-53cb-4493-a98b-261aff790429 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.428996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08b7fe-53cb-4493-a98b-261aff790429 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.429276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08b7fe-53cb-4493-a98b-261aff790429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.435977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fc69-90a2-48d8-b4e5-78406274e2cc None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.436264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fc69-90a2-48d8-b4e5-78406274e2cc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.436545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fc69-90a2-48d8-b4e5-78406274e2cc None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.438236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 463/914] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:20.443196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117488-72e3-4450-9e91-bec303d349ee None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.443478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117488-72e3-4450-9e91-bec303d349ee None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.443750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117488-72e3-4450-9e91-bec303d349ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.448607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2e08b7fe-53cb-4493-a98b-261aff790429 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:20.449793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2e08b7fe-53cb-4493-a98b-261aff790429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:20.462610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2c117488-72e3-4450-9e91-bec303d349ee None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:20.463989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2c117488-72e3-4450-9e91-bec303d349ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:20.538295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 464/915] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:20.545400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 452/916] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:20.719403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15b68-877b-4042-b748-d0b9a7d9056f None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.719760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15b68-877b-4042-b748-d0b9a7d9056f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.719959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15b68-877b-4042-b748-d0b9a7d9056f None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.721581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 465/917] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:20.726830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.727267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.727437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.745905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:20.747118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:20.809082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:20.818260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 453/918] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:20.819371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb15b68-877b-4042-b748-d0b9a7d9056f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:20.829203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:20.886186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:20.945321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb15b68-877b-4042-b748-d0b9a7d9056f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:20.954697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e934d-33af-4164-854d-859ac6142540 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:20.989151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:20.989850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.990385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.990664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.996073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:20.996971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:20.997248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:20.997520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:20.998109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:20.999987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.002202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:21.007695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:21.008834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:21.009837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 466/919] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:21.011206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:21.012342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 03:37:21.013366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:21.014834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.017939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 454/920] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:21.022287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.097699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.125866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17865cb-e3ae-47d6-8f32-bc732144b3e7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.137477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.144680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45583-925f-4700-bb45-f88343ba62cd None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.163892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.164505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.164861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.165555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.166224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.166723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.168179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.176544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 467/921] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:21.179879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.180723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.181254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.181705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.181856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.182650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.183163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.184712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.190288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 455/922] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:21.191417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.295957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9871c-7d62-4ad7-895e-97cdc36e96fe None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.305351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e180cf-5f13-46cf-9c27-9dcb9a6a6166 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.332026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.333433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.334284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.334736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.335174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.335916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.337290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.340875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.341509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.341787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.342057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:21.346554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:21.347551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:21.348448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 468/923] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:21.349240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.359006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.360501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:21.375276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:21.376216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:21.377100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:21.377100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:21.377100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:21.377100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 456/924] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:21.377319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.392929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.464857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0668-65f1-4c0b-a088-b3ebef6f3426 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.480308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97fee-cea2-4316-9ad1-c71b669bec70 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.501542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.502199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.502484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.502750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.503455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.503759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.505127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.510279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 469/925] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:21.511024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.518567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.519585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.519585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.519876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.520474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.520772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.522279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.533910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 457/926] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:21.533910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.580561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.611212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.614370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54830-615d-4751-991a-970d15f6c1d3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.638647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f24813-efa2-4021-9171-051f958a9bb8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.650199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.650858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.651118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.651391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.652035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.652341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.653715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.670289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 470/927] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:21.670289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.680250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.681073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.681414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.681758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.682427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.683928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.704946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 458/928] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:21.705912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.806309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b6213-0cfa-41e1-ae07-c3b6cf9c53b8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.828336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.838103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f81427-41c7-4485-a8d8-fecce5f58cc3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:21.851846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.852482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.852779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.853051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.853687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:21.854026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.856075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5b52c-0992-47e4-ab2d-c94a88f74b16 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.861970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 471/929] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:21.863586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7a3-ddf2-4d06-be53-9766e5f48f0e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.863586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7a3-ddf2-4d06-be53-9766e5f48f0e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.863775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7a3-ddf2-4d06-be53-9766e5f48f0e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.866842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 472/930] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:21.868076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c7a3-ddf2-4d06-be53-9766e5f48f0e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:21.886729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832107-b213-483f-9330-148252693146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:21.887545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832107-b213-483f-9330-148252693146 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.888137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832107-b213-483f-9330-148252693146 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.888426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832107-b213-483f-9330-148252693146 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.894635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:21.897666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832107-b213-483f-9330-148252693146 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:21.899446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832107-b213-483f-9330-148252693146 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a4832107-b213-483f-9330-148252693146 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:21.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:21.913194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:21.914139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 459/931] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:21.915281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:21.915281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:21.915450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:21.933371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:21.934549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:21.984449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c7a3-ddf2-4d06-be53-9766e5f48f0e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.002536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 460/932] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:22.003740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.023563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.024181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.024492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.024831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.025623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] RBAC: Adding query filter params (interface=public, service_id=553df97bb0254761a23e396fbc2c8bb7, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.025934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.027432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.037389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 473/933] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:21 2022] GET /identity/v3/endpoints?service_id=553df97bb0254761a23e396fbc2c8bb7&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:22.038960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.113504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.129613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033628f-1a2e-4c95-9a52-095b7f74d286 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.143681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.148803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd369-3d73-4836-ad89-8d50ae1c17fa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.164313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.164823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.165098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.165365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.166145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.166505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.167967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.187562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 461/934] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:22.188506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.189153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.189426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.189692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.190292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.191944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.192946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.201485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 474/935] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:37:22.286376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.312048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03cfd8-7c49-4638-bb8c-dbac235fa20b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.354315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.355302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.355574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.355841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.366717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.368649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.371115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:22.389372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:22.390377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:22.391402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:22.391402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:22.391402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:22.391580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 462/936] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:22.397849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.419266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b645-935e-4421-9860-b0564e137da8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.469614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.470289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.470615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.470841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.471711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.473329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:22.489421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:22.490418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 03:37:22.491731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:22.494867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 475/937] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:22.496688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.513418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299278ac-6530-41af-b6b1-a6065facbb6a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.521853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.551545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.552159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.552480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.552753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.553396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.553697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.555218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.572936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 463/938] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:22.573861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.630261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd248c-5966-4000-ac1a-24af933a48b0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.640075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.667860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.668288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.668568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.668930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.669572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.669878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.671501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.677208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 476/939] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:22.683779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.695495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096bc31-b57a-44a7-9c37-c4a8b677c146 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.731920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.732969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.733501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.733969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.741492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.743281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:22.755386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:22.756297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:22.757181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:22.759289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 464/940] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:22.765027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.776808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.802536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e366ff-c7ed-415e-aa8a-efcf2269f1fd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.840658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.841287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.841566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.841839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.842477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.842780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.844429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.854514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 477/941] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:22.856896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.880739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee332-3e73-40a8-925c-4e86935ee40e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:22.898255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:22.917805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:22.918792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:22.919292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:22.919803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:22.920681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:22.921194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:22.922993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:22.936821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 465/942] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:22.944507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:22.976257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7808-30ce-4f35-948b-32b602c1730e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:23.018195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:23.018834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:23.019124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:23.019394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:23.020293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:23.022314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52adc-9006-41bd-acc8-0d1c3df6b125 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:23.042930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:23.054901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 478/943] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:23.083014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1079-1daa-4740-8d74-855ffb5b0bbc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:23.130559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:23.131786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:23.132338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:23.132806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/156bd0c3f4b8489888de3ca7b7342e09/roles/5bec5ed56d5a4a44978a688ccd60fb61` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:23.135418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=156bd0c3f4b8489888de3ca7b7342e09, role_id=5bec5ed56d5a4a44978a688ccd60fb61)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:23.137245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bff43-fbce-4d0b-b66d-877c5dd7c871 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:23.153570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 466/944] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:22 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/156bd0c3f4b8489888de3ca7b7342e09/roles/5bec5ed56d5a4a44978a688ccd60fb61 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:23.676145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474902-cd6f-4067-bee5-6f6692da2705 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:23.676682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474902-cd6f-4067-bee5-6f6692da2705 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:23.676682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474902-cd6f-4067-bee5-6f6692da2705 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:23.678235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 479/945] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:23.683627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6aa51-fa81-4be4-9ee7-bb543ac33958 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:23.683909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6aa51-fa81-4be4-9ee7-bb543ac33958 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:23.684177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6aa51-fa81-4be4-9ee7-bb543ac33958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:23.702333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3df6aa51-fa81-4be4-9ee7-bb543ac33958 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:23.703647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3df6aa51-fa81-4be4-9ee7-bb543ac33958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:23.731295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:23.774275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 467/946] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:24.038978 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474902-cd6f-4067-bee5-6f6692da2705 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:24.142797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474902-cd6f-4067-bee5-6f6692da2705 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:24.169262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a3a7-5313-4931-a5c9-a904b88941aa None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.169792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a3a7-5313-4931-a5c9-a904b88941aa None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.169792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a3a7-5313-4931-a5c9-a904b88941aa None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.171527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 468/947] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:24.177315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.177646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.177862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.182631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:24.183584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.184058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.184624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.185603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:24.187271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:24.190145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING keystone.server.flask.common [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 03:37:24.196445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:24.197640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:24.209873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.sql.core [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 03:37:24.209873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 03:37:24.209873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [parameters: {'id': '18aa9d3fc5bc4c90886660cb677377fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 03:37:24.209873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93068) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 18aa9d3fc5bc4c90886660cb677377fb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 18aa9d3fc5bc4c90886660cb677377fb. Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:37:24.228394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application result.read() Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 03 03:37:24.229289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application yield resource Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise self.value Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application session.commit() Sep 03 03:37:24.230564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self.session.flush() Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application compat.raise_( Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 03:37:24.231490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 03:37:24.232394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application result.read() Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:37:24.235298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [parameters: {'id': '18aa9d3fc5bc4c90886660cb677377fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.237501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:24.239474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 18aa9d3fc5bc4c90886660cb677377fb. Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 480/948] 10.222.0.25 () {60 vars in 1140 bytes} [Sat Sep 3 03:37:24 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:24.241616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:24.283735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 469/949] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:24.324952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:24.373239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf9c31-b126-455f-ad24-8fa3232fc9fa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:24.407703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:24.408671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.409148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.409610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.416654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:24.418575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:24.430163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:24.431074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:24.431975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.434284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 481/950] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:24.443925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:24.531217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:24.540131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c48ef3-222c-4eb6-894e-fd91eb83a1c3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:24.574116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:24.574826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.575222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.575536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.576332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:24.576636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:24.577491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:24.578183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:24.604242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 470/951] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:24.702832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356884-d8cf-449f-8755-68be6bd3a5a6 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:24.721241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:24.741155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:24.742037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.742417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.742785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.743554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:24.745062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:24.754979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:24.756174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:24.757078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:24.758711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 482/952] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:24.763996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:24.872935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7971e5-7192-4216-a782-cd4843dd2912 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:24.909210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:24.909836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:24.910169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:24.910445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:24.911164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:24.911472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:24.912865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761df8e-63be-4d44-b608-8f3ae9b44de6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:24.918063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 471/953] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:24.923898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.026769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a5d2-a5c1-437f-849a-47b84e49f920 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.030415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df38335-6b39-49d1-a7e1-135d58329380 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.030700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df38335-6b39-49d1-a7e1-135d58329380 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.031025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df38335-6b39-49d1-a7e1-135d58329380 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.032665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 472/954] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:25.037958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5de19-3884-4b83-8acc-ebb2f9da0b67 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.038244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5de19-3884-4b83-8acc-ebb2f9da0b67 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.038514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5de19-3884-4b83-8acc-ebb2f9da0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.059824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b7c5de19-3884-4b83-8acc-ebb2f9da0b67 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:25.061021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b7c5de19-3884-4b83-8acc-ebb2f9da0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:25.068666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.070137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.070641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.071165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.072480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:25.072987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.073766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.076429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.088447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 483/955] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:25.102838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.150127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.153086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 473/956] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:25.205823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12c64-2273-4cdc-8a67-8a0e73eb9c05 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.206457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12c64-2273-4cdc-8a67-8a0e73eb9c05 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.206457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12c64-2273-4cdc-8a67-8a0e73eb9c05 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.208299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 474/957] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:25.213263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.213648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.213784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d177dc-04ff-4789-90a7-0cb82a5af5a0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.214101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.233246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:25.234670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:25.254511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.255477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.255901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.256530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.257341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.258908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.283845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 484/958] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:37:25.322393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 475/959] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:25.419714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.522252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.526027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180097-fdbb-4830-b483-1fc1b1f6cc2e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.562999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.563632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.563955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.564223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.564808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.566178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:25.574987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:25.575958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:25.576799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:25.581034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 485/960] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:25.582404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.595403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.660349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.700348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f6fd0-e147-4d9f-ad22-b2817ba6c253 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.716112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9924eb2-2294-43d9-a58f-e03ad86a6804 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.748120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.749343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.749630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.750017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.750657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:25.751008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.752401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.753783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.754516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.754928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.755291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.757811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 476/961] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:25.764529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.774881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:25.783093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:25.784045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:25.784883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:25.792873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 486/962] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:25.794541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:25.822124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.896435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62603552-f266-4066-8a50-16f3d2c4a9e3 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.916235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695f836-18e1-40ff-9ab8-f1251767d3df None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:25.945575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.946190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.946465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.946977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.947592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:25.948987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.950426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.952775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:25.962847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 477/963] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:25.965021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:25.965664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:25.965940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:25.966209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:25.966932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:25.967238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:25.968673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:25.978542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 487/964] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:25.984603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:26.100397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c76f97-74d7-4561-96d9-b5f8758e5bf7 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:26.136596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:26.137677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.138176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.138681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.149745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:26.151535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:26.166625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:26.167689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:26.168703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:26.168703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:26.168703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:26.170765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 478/965] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:26.177022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:26.267970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:26.284131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b139c-f86a-406c-9d10-7c557b57d587 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:26.318681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:26.319339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.319995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.320268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.320917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:26.321222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:26.322591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:26.342532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 488/966] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:26.342711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:26.407955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:26.458410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285799c-eb0b-4706-9088-e85155f39969 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:26.494140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:26.494872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.495154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.495423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.504804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:26.506344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:26.518051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:26.519993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:26.521249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 479/967] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:37:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:26.525060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:26.628087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4501f-6735-44e9-9c5d-3c09d455f6ae None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:26.662542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:26.663210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.663495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.663804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.664600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:26.664932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:26.666364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a933a-0a47-4416-8cb1-c5a85a217d6c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:26.684614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 489/968] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Sep 3 03:37:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:26.690270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:26.728076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0241f-6db1-435a-a672-64577b987531 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.728746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0241f-6db1-435a-a672-64577b987531 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.728746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0241f-6db1-435a-a672-64577b987531 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.730383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 490/969] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:26.735459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.737481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.737481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.760015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:26.761531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:26.775202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:26.799259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc7e2e-2cbd-476f-83d5-7782695c922e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:26.815677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:26.838609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:26.839639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:26.840133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:26.840603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:26.854860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 491/970] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:26.915859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:26.917331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.020548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 480/971] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:27.026119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.118669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.134099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.137115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146893a9-f5f5-4ccc-b0e3-cab86aeafb8b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.174850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.175890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.176372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.176834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.179184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.190199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.192118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:27.209244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:27.210229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:27.211169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:27.211169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:27.211169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:27.213306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 492/972] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:27.219686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.243953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954403ad-6db2-4d2a-bc79-0464d492ccb8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.288870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.289799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.290281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.290758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.291591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.293092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.295868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING keystone.server.flask.common [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 03:37:27.313829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.sql.core [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 03:37:27.313829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 03:37:27.313829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [parameters: {'user_id': '4edb6afdaba8484ba87b82be48b135da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 03:37:27.313829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93069) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result.read() Sep 03 03:37:27.323983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:37:27.324931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application yield resource Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application next(self.gen) Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise self.value Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application session.commit() Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 03:37:27.325911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.session.flush() Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application compat.raise_( Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 03:37:27.327162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 03:37:27.328131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application conn.query(q) Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application result.read() Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 03:37:27.329016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [parameters: {'user_id': '4edb6afdaba8484ba87b82be48b135da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:27.329855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 03 03:37:27.335675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 481/973] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Sep 3 03:37:27 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:27.336788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.341425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c643-8f36-4005-a24c-a12da0b26d9c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.378643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.379289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.379567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.379837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.380479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:27.380782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.382145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a612c-ec3c-4726-9a1d-9cc1a96f0bda None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.399485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 493/974] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:27.400679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7bd-4557-4c89-94bf-d8be8fb74947 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.400969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7bd-4557-4c89-94bf-d8be8fb74947 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.401250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7bd-4557-4c89-94bf-d8be8fb74947 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.402795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 494/975] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:27.403530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239e7bd-4557-4c89-94bf-d8be8fb74947 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.423040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.449270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9fd4c-038a-4f39-83d8-d0f1720d5e19 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.454846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.484218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.484939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.485279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.485633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.492974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.494681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-329e1835-d885-4907-bb9d-7b1283a35f3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:27.507460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:27.508638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 03:37:27.509806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:27.515228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 482/976] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:27.515228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.515228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.515228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.516920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239e7bd-4557-4c89-94bf-d8be8fb74947 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.530679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:27.532346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:27.553423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.553984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.554264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.554537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.561489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.563015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:27.574159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:27.575270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:27.576422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:27.580205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 495/977] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:27.581025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.610828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 483/978] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:27.610828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.683431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b0a6-af87-49e5-adff-97c144d3f40b None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.691186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.691578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:27.721120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.722114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.722401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.722680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.723427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:27.723734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.725168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.738530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa5ef-ceb1-45c7-8124-624c5fcb32d8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.771310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 496/979] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:27.786755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.787692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.788148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.788550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.789512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:27.789966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.791639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61759d3b-1e60-4bd1-bc31-d9335322ac39 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.810413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 484/980] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:27.816238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:27.905793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640db4-b796-40d8-a98d-19a9da63f5c3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.906429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640db4-b796-40d8-a98d-19a9da63f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.906934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640db4-b796-40d8-a98d-19a9da63f5c3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.908874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 485/981] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:27.914793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.915191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.915569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.935102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:27.936286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:27.939630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299145d3-9919-4fcb-980b-0795fc19b226 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:27.981051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:27.981765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:27.982046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:27.982323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:27.983093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] RBAC: Adding query filter params (role.id=5bec5ed56d5a4a44978a688ccd60fb61, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=156bd0c3f4b8489888de3ca7b7342e09) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:27.983403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:27.985201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:27.992483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 497/982] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/role_assignments?user.id=156bd0c3f4b8489888de3ca7b7342e09&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=5bec5ed56d5a4a44978a688ccd60fb61 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:27.993098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.010689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 486/983] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:28.030225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:28.116532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208a3a5-eec6-48fe-b7cf-7550b860bcf8 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.157195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.157842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.158127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.158408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.159024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.160404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:28.170551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:28.171714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:28.172598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 498/984] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:28.177767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.237808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:28.282515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.287427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dd921-4b8f-411b-b021-03f57ddcc774 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.321819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.322427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.322700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.322995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.323623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:28.323920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.325290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.336310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 487/985] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:28.337179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.385395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:28.397054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cd03-c966-4ad0-920c-f7557bbc0c1c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.436291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.436903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.437216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.437490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.438142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.439550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.447455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb31-158e-4332-ab74-bf5190e97b9d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.455648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 499/986] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:28 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:37:28.468511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:28.499917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.500552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.500834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.501106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.501758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:28.502066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.503529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.514306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 488/987] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:28.515606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.626390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00304-c5a4-4f94-ab15-dd34ca6a9f28 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.660734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.661249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.661528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.661802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.662520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] RBAC: Adding query filter params (interface=admin, service_id=553df97bb0254761a23e396fbc2c8bb7, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:28.662916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.664467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d688e-f7e5-446e-b381-465b8bd7c558 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.669827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 500/988] 10.222.0.25 () {56 vars in 1249 bytes} [Sat Sep 3 03:37:28 2022] GET /identity/v3/endpoints?service_id=553df97bb0254761a23e396fbc2c8bb7&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:28.675440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:28.772610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038caa09-6520-4ca6-b4c1-5cae53907363 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:28.807929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:28.808531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:28.808822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:28.809093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:28.809697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:28.811112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a975-fc5f-4ac6-af89-56426de1acc5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:28.817622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 489/989] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:28 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:37:29.931396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425bc8-b6db-4fa4-89e7-23b4f5e8b2b0 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:29.931981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425bc8-b6db-4fa4-89e7-23b4f5e8b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:29.931981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425bc8-b6db-4fa4-89e7-23b4f5e8b2b0 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:29.933474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 501/990] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:29.938479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9921-0be6-442e-a2c7-d7c09324ba83 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:29.938933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9921-0be6-442e-a2c7-d7c09324ba83 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:29.939028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9921-0be6-442e-a2c7-d7c09324ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:29.957988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3f8a9921-0be6-442e-a2c7-d7c09324ba83 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:29.959232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3f8a9921-0be6-442e-a2c7-d7c09324ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:29.996551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:30.030465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 490/991] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:30.294615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04425bc8-b6db-4fa4-89e7-23b4f5e8b2b0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:30.314681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:30.389301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdd0cb-a6f0-4bc4-8bdf-e937573b5b27 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.389646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdd0cb-a6f0-4bc4-8bdf-e937573b5b27 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.389861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdd0cb-a6f0-4bc4-8bdf-e937573b5b27 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.391461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 491/992] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:30.396395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.396925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.397307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.405665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04425bc8-b6db-4fa4-89e7-23b4f5e8b2b0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:30.415467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:30.416724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:30.440845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:30.441471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.441752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.442066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] PATH_INFO: `/v3/services/image` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.442590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:30.443974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:30.452974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:30.453963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:30.454847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 502/993] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Sep 3 03:37:30 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:30.459862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:30.494849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 492/994] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:30.546833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:30.567223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df307-674c-45ec-81ef-a68383f707aa None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:30.602316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:30.602952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.603239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.603506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.604133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] RBAC: Adding query filter params (name=image) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:30.604751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:30.606116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e04229-9ad7-4c28-b738-c6cfda60f35f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:30.611477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 503/995] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:30.617269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:30.702714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a7f0-0b39-472d-9771-0718f6900275 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.703469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a7f0-0b39-472d-9771-0718f6900275 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.703755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a7f0-0b39-472d-9771-0718f6900275 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.705245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 504/996] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:30.710249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.710535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.710843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.720999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb92508-ac93-417c-ba56-96e767bc0a16 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:30.722942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:30.733257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:30.734796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:30.757897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:30.758503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.758775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.759070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.759692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] RBAC: Adding query filter params (type=image) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:30.759987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:30.761354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:30.765882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 493/997] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:30.766629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:30.807198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 505/998] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:30.894416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5a56-8cbd-41ef-86cd-8b7b100c1556 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:30.933186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:30.933817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:30.934089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:30.934353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:30.934979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:30.936353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:30.945285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:30.947078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:30.948272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 494/999] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:30.952488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.028888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.055733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166785ec-37e3-40ec-a2b7-414e8d9b1e40 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.073375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.093844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.094507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.094785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.095112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.095783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:31.096599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.098236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.105335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.106135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 506/1000] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:31.110965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.184055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c16d-1382-4da2-8af5-0bb06b12a697 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.219768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec93-9e2d-41f3-8b26-7a13e740d1ce None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.222038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.222986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.222986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.223262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.223823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.225206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:31.238796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:31.239885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:31.240759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 495/1001] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:31.246667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.268827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.269752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.270037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.270305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.270964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:31.271265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.273105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.278352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 507/1002] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:31.283907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.286418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.351759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.359907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47451a-c464-4080-8177-ebcd6d466e96 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.389188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94b04-970e-40b6-9b92-0f9493f229d4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.394898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.395513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.395788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.396060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.396809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:31.397112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.398620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.406850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 496/1003] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:31.409552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.427131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.427734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.428014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.428284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.428988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] RBAC: Adding query filter params (interface=public, service_id=dc49d6e73ba647b2a6cead2bf39fbb9d, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:31.429283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.430673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.435577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 508/1004] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/endpoints?service_id=dc49d6e73ba647b2a6cead2bf39fbb9d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:31.441374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.506166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.516341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2bc5a-8692-4854-b048-b298888917a4 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.543338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6071-b523-4ae2-b3a2-e6ee86eae31e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.551084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.551699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.551976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.552245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.552872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:31.553170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.554548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.559621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 497/1005] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:31.565080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:31.580355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.581188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.581474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.581748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.582377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.583721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04fc6d-0587-46ea-b35b-10a1eff89cef None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.590049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:31.590182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 509/1006] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:31 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:37:31.676880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607287b-1b83-4671-b53c-e99808878d69 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:31.713612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:31.714231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:31.714526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:31.714785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:31.715527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:31.716934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc732c6-3859-4d1e-8d6f-8956dcbfe322 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:31.736377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 498/1007] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:37:32.649453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b63d5-e13e-4a23-b7cd-a572bfa8016e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:32.649983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b63d5-e13e-4a23-b7cd-a572bfa8016e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:32.649983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b63d5-e13e-4a23-b7cd-a572bfa8016e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:32.651463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 510/1008] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:32.656612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f89149-fa05-40f0-a7f3-a8d3a3efe9ce None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:32.657067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f89149-fa05-40f0-a7f3-a8d3a3efe9ce None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:32.657144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f89149-fa05-40f0-a7f3-a8d3a3efe9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:32.677873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-52f89149-fa05-40f0-a7f3-a8d3a3efe9ce None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:32.679148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-52f89149-fa05-40f0-a7f3-a8d3a3efe9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:32.751930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 499/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:33.016804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b63d5-e13e-4a23-b7cd-a572bfa8016e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:33.123598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b63d5-e13e-4a23-b7cd-a572bfa8016e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:33.153892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:33.160502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:33.161118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:33.161649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:33.161934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:33.162557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:33.163967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace85a3-eeb4-475f-900a-4f24870c1888 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:33.186830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 511/1010] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:33 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:33.495145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb98e-df9f-4119-8227-27d6bbf3f310 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:33.495571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb98e-df9f-4119-8227-27d6bbf3f310 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:33.495673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb98e-df9f-4119-8227-27d6bbf3f310 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:33.497192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 500/1011] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:33.502291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878702dc-f668-4d0a-938c-794d789e889f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:33.502540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878702dc-f668-4d0a-938c-794d789e889f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:33.502837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878702dc-f668-4d0a-938c-794d789e889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:33.521482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-878702dc-f668-4d0a-938c-794d789e889f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:33.522668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-878702dc-f668-4d0a-938c-794d789e889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:33.597528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 512/1012] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:33.620466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023b7f-5700-43e3-936b-6eee91bebaa1 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:33.620802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023b7f-5700-43e3-936b-6eee91bebaa1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:33.620972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023b7f-5700-43e3-936b-6eee91bebaa1 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:33.622500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 501/1013] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:33.627587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:33.627876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:33.628118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:33.646703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:33.647898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:33.715802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 513/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:33.861621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023b7f-5700-43e3-936b-6eee91bebaa1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:33.873126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:33.965611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023b7f-5700-43e3-936b-6eee91bebaa1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:33.977902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:34.004694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.005338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.005616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.005880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.006455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.007854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:34.016491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:34.017533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 03:37:34.018373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:34.022513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 502/1015] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:34.023375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:34.027526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:34.080209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a119e8-2eba-483d-a3b2-669707e9fcaf None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:34.095175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:34.115075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.115643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.115923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.116196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.117304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.118734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.121563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 514/1016] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:34.127095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:34.134052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3388-e53a-4f9d-ac3e-84b8c063d966 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:34.170676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.171367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.171652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.171923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.172552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] RBAC: Adding query filter params (name=compute) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:34.172855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.174298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.182837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 503/1017] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:34.185267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:34.239582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c366a4-fe0e-401d-b215-e42c82f597a9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:34.258827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:34.279963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.280611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.280893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.281165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.281814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.283275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.305645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219abe-7a49-4a7f-9fb5-eab28ba00b8a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:34.309562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 515/1018] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:37:34 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:34.344694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.345319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.345593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.345860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.346497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] RBAC: Adding query filter params (type=compute) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:34.346794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.348248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692def-727e-4fe7-a7b5-ad13aecb3e7f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.351171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:34.359221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 504/1019] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Sep 3 03:37:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:34.361282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:34.463591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d21cde-bef7-4a2d-b558-597e1f7e3448 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:34.505234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:34.505859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:34.506140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:34.507161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:34.507864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:34.509260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1db4fb-b9e5-42c3-9570-30dce44ab155 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:34.538768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 516/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:35.085463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbad95-1987-4cae-9333-ed01127e5d66 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:35.086070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbad95-1987-4cae-9333-ed01127e5d66 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:35.086070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbad95-1987-4cae-9333-ed01127e5d66 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:35.087594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 505/1021] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:35.092758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef917-9064-4230-b406-f65df68a6629 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:35.093225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef917-9064-4230-b406-f65df68a6629 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:35.093323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef917-9064-4230-b406-f65df68a6629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:35.112446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-32fef917-9064-4230-b406-f65df68a6629 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:35.113625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-32fef917-9064-4230-b406-f65df68a6629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:35.131730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:35.190770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 517/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:35.449284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbad95-1987-4cae-9333-ed01127e5d66 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:35.554996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbad95-1987-4cae-9333-ed01127e5d66 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:35.587914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:35.588596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:35.588873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:35.589136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] PATH_INFO: `/v3/services/image` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:35.589715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:35.591114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:35.599976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:35.600946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:35.601791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 506/1023] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Sep 3 03:37:35 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:35.609030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fef917-9064-4230-b406-f65df68a6629 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:35.712154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fef917-9064-4230-b406-f65df68a6629 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:35.748069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:35.748676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:35.748947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:35.749215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:35.749920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] RBAC: Adding query filter params (name=image) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:35.750222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:35.751606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:35.761087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 518/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:35.762253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:35.824404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:35.863114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f91d3-d453-4c3e-8c7f-31eafc526834 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:35.896912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:35.897774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:35.898248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:35.898692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:35.899507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] RBAC: Adding query filter params (type=image) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:35.900001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:35.901589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:35.906990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 507/1025] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:35.912759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:35.928776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:36.020947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d110eba-fdd6-47d0-ba76-2d36332a9fe2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:36.054033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:36.054638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.054941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.055215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.055905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] RBAC: Adding query filter params (interface=public, service_id=4b295efb2566450f94b3533f57c0a51f, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:36.056207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:36.057552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7217383-3733-4083-83dd-8472f3731ea6 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:36.062762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 519/1026] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:37:35 2022] GET /identity/v3/endpoints?service_id=4b295efb2566450f94b3533f57c0a51f&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:36.068230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:36.167124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649173a-3ac6-4e7e-ab83-8b3ba66d3396 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:36.193154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:36.202579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:36.203231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.203504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.203752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.204369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:36.205739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46616762-324b-4fda-be68-c1d18c88dbc5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:36.212806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 508/1027] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:37:36 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:37:36.247671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6aaf-cee3-459d-b1af-78a6633557c5 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.248222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6aaf-cee3-459d-b1af-78a6633557c5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.248222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6aaf-cee3-459d-b1af-78a6633557c5 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.249834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 520/1028] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:36.256580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75df1-f414-4e8f-8a74-557762d8873c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.256973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75df1-f414-4e8f-8a74-557762d8873c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.257157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75df1-f414-4e8f-8a74-557762d8873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.277211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-84f75df1-f414-4e8f-8a74-557762d8873c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:36.278332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-84f75df1-f414-4e8f-8a74-557762d8873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:36.355895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 509/1029] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:36.541551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66dad-e470-421a-9a4c-984cbc1236bf None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.541760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66dad-e470-421a-9a4c-984cbc1236bf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.542051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66dad-e470-421a-9a4c-984cbc1236bf None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.543587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 521/1030] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:36.548763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.549120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.549463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.568236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:36.569421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:36.617688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b66dad-e470-421a-9a4c-984cbc1236bf None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:36.643335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 510/1031] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:36.696554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:36.721399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b66dad-e470-421a-9a4c-984cbc1236bf None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:36.754740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:36.755396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.755672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.755938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.757029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:36.758453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:36.760238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 522/1032] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:36.765839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:36.816944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:36.863397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581a42a-7e36-43c8-955f-bd0ea75af4bc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:36.897075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:36.897677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:36.897952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:36.898189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:36.898860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:36.900313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:36.922746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 511/1033] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:37:36 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:36.923907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.037407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016962-f65e-416c-9942-c3b6eb7cc4c5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.074825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.075401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.075685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.075963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.087070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:37.091862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:37.092782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:37.093782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 523/1034] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:37.099600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.203863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bb501-bcb7-47b4-a25d-ac7e7e424f3f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.213520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:37.244304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.244923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.245197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.245464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.246127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:37.246425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:37.247830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:37.262196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 512/1035] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:37.263694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.267944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:37.369421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcfec3-e0d5-4d31-98f9-95e7a5e1785e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.404585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.405219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.405523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.405792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.416315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:37.417748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:37.432092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:37.433084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:37.434008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:37.434008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:37.434008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:37.434008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 524/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:37.440218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.541035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ad3e-90c4-4c96-9b7a-cd7da69742df None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.573974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.574579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.574871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.575142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.575774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:37.576079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:37.577489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:37.589497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 513/1037] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:37.595133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.650942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:37.696913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545cf4f-c668-426a-b37f-bc1634f1858e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.730431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.731091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.731364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.731628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.739115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:37.740556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:37.751619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:37.752574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:37.753526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:37.754887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 525/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:37.758503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:37.789557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:37.859687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e6d8e-cd6f-4543-af62-02be6dd071f5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:37.892687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:37.893291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:37.893564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:37.893830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:37.894556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:37.894883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:37.896288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16befd7-38a7-4812-9b90-4cdfdc0fe105 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:37.913516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 514/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:37.919101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.018521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a42b4-2203-415d-972e-1065780a14cc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.034530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.055952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.056566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.056836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.057105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.078885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12802-4094-4572-aeb3-1bcdac1188fd None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.079159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12802-4094-4572-aeb3-1bcdac1188fd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.079428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12802-4094-4572-aeb3-1bcdac1188fd None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.080898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 515/1040] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:38.085431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.085718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.086005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.103640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:38.105118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:38.125952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.127589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.173513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 516/1041] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:38.233856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 526/1042] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:38.236001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.260030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.353838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e1fd-0f98-4daf-a992-081fc8c963b4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.389455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.390211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.390484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.390747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.401519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.403000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:38.417827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:38.418790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:38.419723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:38.419723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:38.419723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:38.423167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 517/1043] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:38.424841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.440919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.444781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.516063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.532345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8fde9-616e-4162-92d0-2243936cdbd4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.551738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f80a-1e4d-4e6a-9bf0-024e56174847 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.566292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.566898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.567174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.567443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.568103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:38.568400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.569765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.581192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 527/1044] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:38.586775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.596348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.596852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.597126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.597391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] PATH_INFO: `/v3/services/image` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.597968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.599373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:38.608486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:38.609489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:38.610315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 518/1045] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:38.615789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.667785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.691955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905eb1-b990-4422-9189-55543b41c648 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.726193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480737-dd35-4b18-8539-b3d535b3f76a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.727962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.728882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.729487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.729756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.737116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.738628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:38.750661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:38.751922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:38.752938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 528/1046] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:38.757627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.763027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.763634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.763885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.764176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.764816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] RBAC: Adding query filter params (name=image) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:38.765143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.766509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.767636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.777720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 519/1047] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:38.778929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.863107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809e900-1e6a-4e42-9403-96434759fbae None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.883097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f82d3-27bc-4b05-a682-031d046aff8e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:38.899424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.900036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.900312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.900580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.901262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:38.901564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.903025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afb612-9dbc-4c7e-9271-18f1e7378c35 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.905459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:38.915471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 529/1048] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:38.916534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:38.916751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ded62-c84c-44dc-8a76-a7284a586940 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.917006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ded62-c84c-44dc-8a76-a7284a586940 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.917164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.917277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ded62-c84c-44dc-8a76-a7284a586940 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.917460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.917672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.918298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] RBAC: Adding query filter params (type=image) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:38.918599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:38.920006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a130-cf13-48d1-9cf8-c11780a89339 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:38.922817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 530/1049] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:38.923631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ded62-c84c-44dc-8a76-a7284a586940 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:38.925058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 520/1050] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:38.926103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:38.926391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:38.926656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:38.944243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:38.945486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:38.999175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:39.010298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 521/1051] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:39.011226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:39.033582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ded62-c84c-44dc-8a76-a7284a586940 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.075693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.076385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.076951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.077316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.078667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] RBAC: Adding query filter params (role.id=5bec5ed56d5a4a44978a688ccd60fb61, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:39.079068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:39.081053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:39.088276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 531/1052] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:38 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=5bec5ed56d5a4a44978a688ccd60fb61 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:39.118155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19001-ac09-4f84-a092-f6c5b2bc56ab None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.157475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.158094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.158369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.158635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.159410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:39.160816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:39.190904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 522/1053] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:37:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:39.281402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:39.345650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:39.392345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa10-d4f8-47d6-8aea-ec43e95a5eb9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.432502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.433257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.433340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.433640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.444884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:39.449675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:39.450626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:39.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 532/1054] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:39.457629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:39.509559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:39.566335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b99e79-de2a-4415-843b-34c45e829dc0 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.600365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.600974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.601251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.601512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.602185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:39.602483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:39.603890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:39.617760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 523/1055] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:39.618728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:39.722733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a1669-1ba5-4706-8a9a-690c70330ac3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.740065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:39.759782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.760333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.760587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.760884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] PATH_INFO: `/v3/users/c67b66bf2bbf437782037b33d4278852` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.774927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] RBAC: Authorizing `identity:get_user(user_id=c67b66bf2bbf437782037b33d4278852)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:39.776368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:39.778154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 533/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:39 2022] GET /identity/v3/users/c67b66bf2bbf437782037b33d4278852 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:39.784228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:39.887246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e878bd-4cc8-4543-b6a6-e4a9f6213ce9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:39.897949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:39.924369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:39.924990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:39.925271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:39.925539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:39.993649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:39.995160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:40.094898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 524/1057] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:40.100409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:40.207074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255372-53b0-4e60-9aee-c2b1417f7216 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:40.245998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:40.246623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:40.246986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:40.247252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:40.248353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:40.249814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48692a-82a8-4f9c-a335-9ce486446487 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:40.256954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 534/1058] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:40 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:40.260817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:40.274564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:40.366897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb9cb-b207-4af5-936f-d5429caa8519 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:40.400026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:40.400628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:40.400903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:40.401171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:40.401894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=a5f75fb694534872be9b074f652582d0, user.id=c67b66bf2bbf437782037b33d4278852) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:40.402212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:40.403688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755b00c-f256-41b6-a4a0-cbacb05bd900 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:40.409276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 525/1059] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:40 2022] GET /identity/v3/role_assignments?user.id=c67b66bf2bbf437782037b33d4278852&scope.project.id=a5f75fb694534872be9b074f652582d0&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:40.965671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8125d78-56fd-4e2e-97b6-bde6fd5209a6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:40.966212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8125d78-56fd-4e2e-97b6-bde6fd5209a6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:40.966212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8125d78-56fd-4e2e-97b6-bde6fd5209a6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:40.967814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 535/1060] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:40.976928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa1f9-12c4-4ee9-a7e9-5b976d29577c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:40.976928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa1f9-12c4-4ee9-a7e9-5b976d29577c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:40.976928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa1f9-12c4-4ee9-a7e9-5b976d29577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:40.995046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-119fa1f9-12c4-4ee9-a7e9-5b976d29577c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:40.996575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-119fa1f9-12c4-4ee9-a7e9-5b976d29577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:41.074910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 526/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:41.113265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62186bf-dfd1-4062-9534-22670dd9bfd7 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.113540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62186bf-dfd1-4062-9534-22670dd9bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.113819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62186bf-dfd1-4062-9534-22670dd9bfd7 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.115430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 536/1062] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:41.120828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.121117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.121394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.128891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.140989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:41.142345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:41.215162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 527/1063] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:41.341694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62186bf-dfd1-4062-9534-22670dd9bfd7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.388955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.450178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62186bf-dfd1-4062-9534-22670dd9bfd7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.480293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.488105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:41.488762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.489044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.489320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.499619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:41.504455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:41.505397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:41.506284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 537/1064] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:41.511332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.584941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920cd3-e27b-4f7a-ac47-f4bcb6a6c75b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.613383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cffe4-906e-4afe-a158-7ee429aee82c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.615248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.620731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:41.621345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.621624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.621897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.624607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.651193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:41.651889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.652077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.652357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.653254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:41.653362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:41.654691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:41.664524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 538/1065] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:41.669898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.697340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:41.698832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:41.768684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e0fa-0813-4ee3-826e-930b2da252ba None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.801799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:41.802431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.802709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.802997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.804112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 528/1066] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:37:41.809625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.818268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:41.819763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:41.834538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:41.835724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:41.836644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:41.836644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:41.836644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:41.836644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 539/1067] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:41.841332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:41.851183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.869468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:41.922434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3149d-59ec-454e-93f1-9494d95b600d None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.960371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:41.961157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:41.961437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:41.961707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:41.971467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831485a-fa5d-4a6d-90ff-6566bee26198 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:41.973114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:41.974584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:41.990648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:41.992099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:41.992986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:41.992986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:41.992986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:41.992986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 529/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:41.997581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.008488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.009107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.009408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.009680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.010325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:42.010626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.012043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.024289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 540/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:42.030351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.085812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.101032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688e74-5145-497d-9b06-4c7ef5e47078 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.111251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.136445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc819a-2a71-4701-b2bd-05b972c64867 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.137543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.138412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.138926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.139262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.139982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:42.140350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.141943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.162845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 530/1070] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:42.174034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.174701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.175012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.175284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.183805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.185358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-ba6f59e3-9a0e-41a5-9613-5a74501dba5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:42.198641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:42.199755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:42.200725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:42.203266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 541/1071] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:42.205939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.288526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6942-7fa9-4909-9108-7c9f06847ff8 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.288772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6942-7fa9-4909-9108-7c9f06847ff8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.289070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc6942-7fa9-4909-9108-7c9f06847ff8 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.290396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 542/1072] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:37:42.295640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.295969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.296273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.318247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:42.319874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:42.323707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d52d-22f7-456b-9081-80dea80cbd1b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.364146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.364702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.364983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.365258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.365996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:42.366302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.367780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.387909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 531/1073] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:42.394535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.395029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 543/1074] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:37:42.398484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.493825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fce340-f794-4ffb-b01b-e7eb19089a67 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.527979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.528604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.528874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.529127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.539196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.540647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:42.555213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:42.556165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:42.557470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:42.557470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:42.557470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:42.557470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 532/1075] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:42.562291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.629604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.657038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.663873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e81a1-98bc-4294-9925-fe50bc4fa98d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.697754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.698748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.699263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.699882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.700997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:42.701517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.704225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.714755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.715576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 544/1076] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:42.721315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.758388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb09ef-308b-40e5-a1df-faf8f243e6be None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.791135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.791768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.792046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.792319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.802489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:42.807454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:42.808579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:42.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 533/1077] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:42.814586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.821237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59c8d-7f76-47f1-8498-2c183b173c38 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.845617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.855128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.855761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.856080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.856315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.862924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.864362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:42.875405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:42.876342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:42.877236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:42.881351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 545/1078] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:42.882095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.914300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa492-54d6-4db6-8868-c0eabda54245 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:42.929955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:42.951124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:42.951730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:42.952011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:42.952313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:42.952973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:42.953276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:42.954680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:42.962694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 534/1079] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:42.968553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:42.981959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a24e0-da1d-41d0-9ebf-be83d437b6cc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:43.014458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:43.015108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:43.015383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:43.015639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:43.016338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:43.016628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:43.018046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:43.029631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 546/1080] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:43.035566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:43.065366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce713d46-209d-4632-bffb-555aeee40c33 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:43.079346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:43.098288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:43.098945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:43.099218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:43.099482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] PATH_INFO: `/v3/users/c67b66bf2bbf437782037b33d4278852` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:43.100589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] RBAC: Authorizing `identity:get_user(user_id=c67b66bf2bbf437782037b33d4278852)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:43.102016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:43.108892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 535/1081] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:42 2022] GET /identity/v3/users/c67b66bf2bbf437782037b33d4278852 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:43.109723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:43.132711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac44b7-4c52-4d0f-8601-a989f3a2ff05 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:43.158163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:43.169567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:43.170230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:43.170515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:43.170883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/5bec5ed56d5a4a44978a688ccd60fb61` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:43.173136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=8e0e6be0fc61491fa25193834b4fa8af, role_id=5bec5ed56d5a4a44978a688ccd60fb61)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:43.174571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:43.187216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 547/1082] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:43 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/8e0e6be0fc61491fa25193834b4fa8af/roles/5bec5ed56d5a4a44978a688ccd60fb61 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:37:43.214969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edf078-5a27-4658-b7a9-d73387a44155 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:43.260275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:43.260936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:43.261214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:43.261483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:43.262704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:43.264222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad216a-3d03-4418-a5b2-c994ffe8011b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:43.266196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 536/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:43 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:43.272621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:43.379757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09670389-13c0-474b-b074-8fd9248fc04d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:43.387280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:43.417932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:43.418594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:43.418919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:43.419180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0/users/c67b66bf2bbf437782037b33d4278852/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:43.421225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f75fb694534872be9b074f652582d0, user_id=c67b66bf2bbf437782037b33d4278852, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:43.422649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b15098-d8b9-430c-9705-759d48a9f585 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:43.436599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 548/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:43 2022] PUT /identity/v3/projects/a5f75fb694534872be9b074f652582d0/users/c67b66bf2bbf437782037b33d4278852/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:45.044829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913946-dff9-47a3-bff0-0188168d11cd None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.045388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913946-dff9-47a3-bff0-0188168d11cd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.045388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913946-dff9-47a3-bff0-0188168d11cd None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.046961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 537/1085] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:45.052180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47cc8-1f5f-486d-9abc-ea2b9a8e6b8f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.052623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47cc8-1f5f-486d-9abc-ea2b9a8e6b8f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.052709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a47cc8-1f5f-486d-9abc-ea2b9a8e6b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.072266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e9a47cc8-1f5f-486d-9abc-ea2b9a8e6b8f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:45.073480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e9a47cc8-1f5f-486d-9abc-ea2b9a8e6b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:45.146410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 549/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:45.280085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000939ba-c349-44fd-bfe7-56f19aa703c6 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.280460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000939ba-c349-44fd-bfe7-56f19aa703c6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.280727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000939ba-c349-44fd-bfe7-56f19aa703c6 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.282082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 538/1087] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:45.287458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.287746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.288014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.306778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:45.307917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:45.374781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 550/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:45.412664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000939ba-c349-44fd-bfe7-56f19aa703c6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.516698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000939ba-c349-44fd-bfe7-56f19aa703c6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:45.530095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:45.551683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:45.552303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.552575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.552841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.563956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:45.568591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:45.569625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:45.570590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 539/1089] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:45.575930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.603674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:45.639634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.678584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a943d-3725-45d7-b755-e17f8a8b27bb None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:45.712901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:45.713507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.713786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.714055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.714714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:45.715043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:45.716416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:45.724241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 551/1090] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:45.730262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.741701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa205a4-d38e-4485-8498-9e0edc0e0d2e None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:45.775309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:45.775954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.776230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.776495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.787142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:37:45.791845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:45.792790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:37:45.793654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 540/1091] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:37:45.799070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.825246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:45.836902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:45.839145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09a60-dfbb-4cf8-87c4-d2a873270eb9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:45.872242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:45.872860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.873131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.873383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.884554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:45.886200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:45.900041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:45.901085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:45.902477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:45.902477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:45.902477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:45.902477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 552/1092] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:45.907030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:45.909197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fb53-3af2-411b-a1cf-5530c5a78364 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:45.942678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:45.943302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:45.943576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:45.943829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:45.944486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:45.944781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:45.946147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:45.954029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 541/1093] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:45.959948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.007059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae631bc-d1cf-48ea-85c8-b9bed1c45a31 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.038866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.039462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.039733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.040001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.040629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:46.040920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.042268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.050665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.052521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.059093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 553/1094] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:46.060106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.063198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a6283-0c53-4b74-bd50-95d4f1156921 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.096827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.097393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.097924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] PATH_INFO: `/v3/users/c67b66bf2bbf437782037b33d4278852` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.099087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] RBAC: Authorizing `identity:get_user(user_id=c67b66bf2bbf437782037b33d4278852)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.100521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.102384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 542/1095] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:45 2022] GET /identity/v3/users/c67b66bf2bbf437782037b33d4278852 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:46.108384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.157488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5d3-7df8-40a6-b441-0735cebcaad3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.191273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.191898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.192174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.192445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.199430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.200858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.204554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b04e1-25df-4cff-a595-5c48077dce82 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:37:46.212079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:46.213159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:46.214107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 554/1096] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:46.219098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.240182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.240796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.241073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.241345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.271395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.279178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.307547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.309035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.319957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c433-b978-4a4d-a73f-74a3a84be43f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.351922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.352458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.352738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.353050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.353762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:46.354383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.355827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.372332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 555/1097] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:46.379938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.409661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 543/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:37:46.418126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.483466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.490233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdc585-4a24-46db-afb0-428af58fe204 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.507258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.520280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7432ad-3dfe-4e60-aec4-ae3ac0c5f194 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.524657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.525263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.525539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.525808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.553967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.554585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.554907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.555174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.556271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.557729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.559393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 544/1099] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:46.565674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.599719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.601223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.669204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e79f6-1bf8-4d1f-b4d1-664633b2635f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.705234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 556/1100] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:37:46.705437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.706485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.708879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.709568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.710032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.710497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.711794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=a5f75fb694534872be9b074f652582d0, user.id=c67b66bf2bbf437782037b33d4278852) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:46.712173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.714186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.720856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 545/1101] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/role_assignments?user.id=c67b66bf2bbf437782037b33d4278852&scope.project.id=a5f75fb694534872be9b074f652582d0&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:37:46.728036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:46.821532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980b10-29e4-4c29-8e4b-1009297c1494 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:46.862201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:46.862829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:46.863153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:46.864167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:46.873681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:46.875242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:37:46.891194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:46.892339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:37:46.893665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:37:46.893665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:37:46.893665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:46.893665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 557/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:46.898313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:46.999332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328b786-ee0b-4728-820a-a16910ab8ec8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:47.033774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:47.034378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:47.034624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:47.034937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:47.035575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:47.035869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:47.037238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:47.049772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 546/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:47.055442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:47.128871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:47.164966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb73bd-7472-44de-b38d-887fb0aca7cf None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:47.203206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:47.203839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:47.204111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:47.204374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:47.211337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:47.212766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:37:47.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:37:47.224965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:37:47.226266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 558/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:37:47.231431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:47.299616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:47.346144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27608f2c-bb12-4dba-b67a-c013adc65903 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:47.381520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:47.382124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:47.382399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:47.382664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:47.383503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:47.383809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:47.385246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d537d7-6555-4350-b628-82661bd400ed None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:47.396807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 547/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:37:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:47.403825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:47.508499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08bceb-ec6e-46a5-a20d-b9ef135063cb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:47.536711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:47.543383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:47.543976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:47.544249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:47.544511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:47.545223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] RBAC: Adding query filter params (role.id=5bec5ed56d5a4a44978a688ccd60fb61, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=8e0e6be0fc61491fa25193834b4fa8af) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:47.545523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:47.547000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ea2b8-a18d-4f0f-ad1c-70602cb646b9 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:47.552929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 559/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:47 2022] GET /identity/v3/role_assignments?user.id=8e0e6be0fc61491fa25193834b4fa8af&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=5bec5ed56d5a4a44978a688ccd60fb61 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:48.642576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321d6fc-0fad-493b-9472-811a31b64b18 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:48.643434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321d6fc-0fad-493b-9472-811a31b64b18 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:48.643434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321d6fc-0fad-493b-9472-811a31b64b18 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:48.645554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 548/1107] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:48.651815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:48.652245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:48.652353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:48.678400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:48.679617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:48.758921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 560/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:49.023209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321d6fc-0fad-493b-9472-811a31b64b18 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:49.129190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321d6fc-0fad-493b-9472-811a31b64b18 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:49.145091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:49.171773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:49.172390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:49.172659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:49.172925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:49.174170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:49.175655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99552e7d-2123-44f5-b190-9d07eb7aab82 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:49.177225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 549/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:37:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:49.183391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:49.291082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d904-9e1b-402d-bd43-423e48630c6a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:49.316252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:49.328627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:49.329224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:49.329498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:49.329747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:49.330387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:49.331829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e74d-e862-4764-a38b-00ca7f52df2e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:49.358577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 561/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:37:49 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:37:51.273984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8b0d-9cfa-46fa-8054-a7ac92a0c19f None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:51.274434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8b0d-9cfa-46fa-8054-a7ac92a0c19f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:51.274527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8b0d-9cfa-46fa-8054-a7ac92a0c19f None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:51.276688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 550/1111] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:51.284374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461681-4601-4282-8455-790682fe92dc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:51.284776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461681-4601-4282-8455-790682fe92dc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:51.284972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461681-4601-4282-8455-790682fe92dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:51.313161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-aa461681-4601-4282-8455-790682fe92dc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:51.314370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-aa461681-4601-4282-8455-790682fe92dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:51.383357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 562/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:51.647290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8b0d-9cfa-46fa-8054-a7ac92a0c19f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:51.755800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8b0d-9cfa-46fa-8054-a7ac92a0c19f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:51.790134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:51.790747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:51.791048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:51.791313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] PATH_INFO: `/v3/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:51.792801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] RBAC: Authorizing `identity:get_role(role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:51.794184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:51.795546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 551/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:51 2022] GET /identity/v3/roles/c566cf204ec94aaba0a8466d98161f8d => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:51.802093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa461681-4601-4282-8455-790682fe92dc None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:51.912800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa461681-4601-4282-8455-790682fe92dc None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:51.923307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:51.953237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:51.953859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:51.954137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:51.954404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] PATH_INFO: `/v3/users/37b37ae90d0d47428e4f30863ff697ba` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:51.967529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] RBAC: Authorizing `identity:get_user(user_id=37b37ae90d0d47428e4f30863ff697ba)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:51.968967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:51.970561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 563/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:51 2022] GET /identity/v3/users/37b37ae90d0d47428e4f30863ff697ba => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:51.976380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:52.032195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:52.080493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d23b7-9c84-4657-aa57-281d34c71edf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:52.117879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:52.118467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:52.118741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:52.119034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:52.187305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:52.188741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:52.242498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:52.290698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 552/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:52.296908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:52.398237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4816-44e7-44c4-9d10-f9021061e491 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:52.436036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:52.436646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:52.436916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:52.437179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:52.438455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:52.439842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054e60-65a1-4a16-94bb-b06c4abbbd33 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:52.441422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 564/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:52 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:52.447542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:52.552073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfe35-3656-45da-98a8-68bea8ebccbb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:52.591814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:52.592694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:52.592970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:52.593235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:52.593957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a5f75fb694534872be9b074f652582d0, user.id=37b37ae90d0d47428e4f30863ff697ba) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:52.594260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:52.595720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc8305-54ab-4e07-b74c-f9f68f837c1a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:52.601632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 553/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:52 2022] GET /identity/v3/role_assignments?user.id=37b37ae90d0d47428e4f30863ff697ba&scope.project.id=a5f75fb694534872be9b074f652582d0&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:54.442081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dcab1-a2dd-4dae-a708-22a3daae5390 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:54.442974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dcab1-a2dd-4dae-a708-22a3daae5390 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:54.442974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dcab1-a2dd-4dae-a708-22a3daae5390 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:54.445202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 565/1118] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:54.451450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:54.451958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:54.452111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:54.475546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:54.477404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:54.490755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:54.553525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 554/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:54.817209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dcab1-a2dd-4dae-a708-22a3daae5390 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:54.857317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:54.922774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dcab1-a2dd-4dae-a708-22a3daae5390 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:54.957648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:54.958259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:54.958532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:54.958822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] PATH_INFO: `/v3/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:54.960323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] RBAC: Authorizing `identity:get_role(role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:54.961680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:54.963010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 566/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:54 2022] GET /identity/v3/roles/c566cf204ec94aaba0a8466d98161f8d => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:54.969657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:55.090061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ec764-c530-40b4-9481-f8aeca068664 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:55.126024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:55.126637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:55.126945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:55.127216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] PATH_INFO: `/v3/users/37b37ae90d0d47428e4f30863ff697ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:55.128378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] RBAC: Authorizing `identity:get_user(user_id=37b37ae90d0d47428e4f30863ff697ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:55.129811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:55.131406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 555/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:54 2022] GET /identity/v3/users/37b37ae90d0d47428e4f30863ff697ba => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:55.140319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:55.250704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cd86-75ad-4350-a893-0d71128f6f8e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:55.253078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:55.286935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:55.287584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:55.287864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:55.288133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:55.289249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:55.290670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7a901-363b-4c78-b897-d2483959065c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:55.292320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 567/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:55 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:55.299385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:55.315395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:55.401030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de292cc7-5ec9-43e2-bc18-0f3df33b7ecc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:55.435280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:55.435947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:55.436217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:55.436479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0/users/37b37ae90d0d47428e4f30863ff697ba/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:55.438778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f75fb694534872be9b074f652582d0, user_id=37b37ae90d0d47428e4f30863ff697ba, role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:55.440237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b70a-a5cd-47ed-b133-d21a0fbb8f82 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:55.455139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 556/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:37:55 2022] PUT /identity/v3/projects/a5f75fb694534872be9b074f652582d0/users/37b37ae90d0d47428e4f30863ff697ba/roles/c566cf204ec94aaba0a8466d98161f8d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:37:57.309625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa7f5-7b57-48d6-af8b-8702cc26e756 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:57.310200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa7f5-7b57-48d6-af8b-8702cc26e756 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:57.310200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa7f5-7b57-48d6-af8b-8702cc26e756 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:57.311657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 568/1124] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:37:57.318077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:57.318856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:57.319010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:57.347377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:37:57.348632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:37:57.423867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 557/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:37:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:37:57.700987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fa7f5-7b57-48d6-af8b-8702cc26e756 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:57.826680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fa7f5-7b57-48d6-af8b-8702cc26e756 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:57.862138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:57.862752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:57.863062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:57.863333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] PATH_INFO: `/v3/roles/c566cf204ec94aaba0a8466d98161f8d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:57.864841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] RBAC: Authorizing `identity:get_role(role_id=c566cf204ec94aaba0a8466d98161f8d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:57.866205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:57.867548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 569/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:57 2022] GET /identity/v3/roles/c566cf204ec94aaba0a8466d98161f8d => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:57.873838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:57.877127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:57.990955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f9411-abd7-470a-9f34-60a4c5fae309 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:58.028358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:58.028972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:58.029246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:58.029581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] PATH_INFO: `/v3/users/37b37ae90d0d47428e4f30863ff697ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:58.030834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] RBAC: Authorizing `identity:get_user(user_id=37b37ae90d0d47428e4f30863ff697ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:58.032282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:58.033841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 558/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:37:57 2022] GET /identity/v3/users/37b37ae90d0d47428e4f30863ff697ba => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:58.039910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:58.082963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:58.146758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9d9b-2ea5-4c2c-b217-62a96de18857 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:58.182576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:58.183235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:58.183510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:58.183783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:58.252639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:58.253984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:58.295409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:58.359328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 570/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:37:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:37:58.365772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:58.438155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:37:58.473773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9219fe-1e46-4610-bc86-16451f1e38e5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:58.509572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:58.510124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:58.510401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:58.510673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] PATH_INFO: `/v3/projects/a5f75fb694534872be9b074f652582d0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:58.511947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] RBAC: Authorizing `identity:get_project(project_id=a5f75fb694534872be9b074f652582d0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:58.513391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3a2d0-ffe0-4f12-8824-8fbf25a66c24 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:58.515086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 559/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:37:58 2022] GET /identity/v3/projects/a5f75fb694534872be9b074f652582d0 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:37:58.521389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:37:58.631478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54939d5-7c21-49f4-8975-d5ed84e37633 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:37:58.676552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:37:58.677147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:37:58.677422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:37:58.677717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:37:58.678430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] RBAC: Adding query filter params (role.id=c566cf204ec94aaba0a8466d98161f8d, scope.project.id=a5f75fb694534872be9b074f652582d0, user.id=37b37ae90d0d47428e4f30863ff697ba) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:37:58.678735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:37:58.680190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f848f23-8088-45ed-a380-367d9e30de79 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:37:58.686431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 571/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:37:58 2022] GET /identity/v3/role_assignments?user.id=37b37ae90d0d47428e4f30863ff697ba&scope.project.id=a5f75fb694534872be9b074f652582d0&role.id=c566cf204ec94aaba0a8466d98161f8d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:00.589737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248c8e-cda3-42d8-b7cd-104b3835f5f0 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:00.590576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248c8e-cda3-42d8-b7cd-104b3835f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:00.590576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248c8e-cda3-42d8-b7cd-104b3835f5f0 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:00.592676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 560/1131] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:00.600545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:00.600998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:00.601096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:00.609719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:00.621281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:00.622467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:00.698930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 572/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:38:00.963243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248c8e-cda3-42d8-b7cd-104b3835f5f0 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:01.074227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248c8e-cda3-42d8-b7cd-104b3835f5f0 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:01.109123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:01.109736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:01.110011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:01.110278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:01.111450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:01.112893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636892e7-22fc-4b85-a30e-657a03217521 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:01.114451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 561/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:38:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:01.120973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:01.236777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14583efe-9a20-4599-9ce1-54ada4dc7d90 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:01.274088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:01.274678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:01.274974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:01.275239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:01.275904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:01.277320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16065b-6a7f-4827-b4fe-54596ee7b84d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:01.282046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:01.308642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 573/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:38:01 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:38:03.200211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd179d-eda7-4d95-bf0c-8fc72d5a41b8 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:03.200797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd179d-eda7-4d95-bf0c-8fc72d5a41b8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:03.200797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd179d-eda7-4d95-bf0c-8fc72d5a41b8 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:03.202116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 562/1135] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:03.208461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:03.209205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:03.209336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:03.231898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:03.233113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:03.306602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 574/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:03.578003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd179d-eda7-4d95-bf0c-8fc72d5a41b8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:03.590507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:03.687311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd179d-eda7-4d95-bf0c-8fc72d5a41b8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:03.731396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:03.732459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:03.732752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:03.733027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:03.734196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:03.735675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985cd0-5be6-498c-abaf-8f64f7cc152d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:03.737253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 563/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:38:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:03.743140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:03.842524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cadee5-b228-4a0d-96d1-d627abc21208 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:03.859175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:03.877924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:03.878515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:03.878790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:03.879084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:03.879720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:03.881137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cc3b1-7707-4897-a08c-c132723ce02c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:03.906122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 575/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:38:03 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:38:05.802432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4ae8-d89b-4916-9405-5fd0046c04c2 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:05.803384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4ae8-d89b-4916-9405-5fd0046c04c2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:05.803384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4ae8-d89b-4916-9405-5fd0046c04c2 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:05.805493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 564/1139] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:05.814300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:05.814732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:05.814889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:05.838285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:05.839509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:05.922740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 576/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:06.186700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb4ae8-d89b-4916-9405-5fd0046c04c2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:06.293340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:06.319456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb4ae8-d89b-4916-9405-5fd0046c04c2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:06.359240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:06.360254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:06.360672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:06.361117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:06.377181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:06.381932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:06.383089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:06.383986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 565/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:06.389571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:06.501263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77722f3f-4d1b-48dc-99d8-339b2d2f28ea None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:06.503584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:06.543572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:06.544176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:06.544453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:06.544785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:06.545460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:06.545769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:06.547163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:06.555551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 577/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:06.565109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:06.675572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eca01-ad8a-401b-89c3-7a9039cf301a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:06.722019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:06.722639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:06.722944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:06.723217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] PATH_INFO: `/v3/users/8745ceb73cfa4825acf3bd38d757659d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:06.726135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:06.737379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] RBAC: Authorizing `identity:get_user(user_id=8745ceb73cfa4825acf3bd38d757659d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:06.738861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:06.740483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 566/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:06 2022] GET /identity/v3/users/8745ceb73cfa4825acf3bd38d757659d => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:06.749352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:06.851281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a01ad-de67-4c33-aa9e-8c728893a967 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:06.886451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:06.887071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:06.887351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:06.887625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:06.919824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:06.957827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:06.959285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:07.057484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 578/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:38:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:38:07.064213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:07.177800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6feaf-0234-4278-a890-8b27ef49e050 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:07.213121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:07.213727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:07.213999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:07.214267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] PATH_INFO: `/v3/projects/eb49414f27cc49e6bc500f4dab137c38` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:07.215524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] RBAC: Authorizing `identity:get_project(project_id=eb49414f27cc49e6bc500f4dab137c38)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:07.216993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75caccf-2686-4687-a66a-657726ba8880 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:07.218593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 567/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:07 2022] GET /identity/v3/projects/eb49414f27cc49e6bc500f4dab137c38 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:07.224883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:07.291996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:07.342423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d68148-dfb7-4cb5-aa54-bce147a14d82 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:07.381949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:07.382546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:07.382849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:07.383119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:07.383804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=eb49414f27cc49e6bc500f4dab137c38, user.id=8745ceb73cfa4825acf3bd38d757659d) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:07.384140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:07.385552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42a00-269a-44e9-9683-1fc0457a1a14 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:07.391301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 579/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:38:07 2022] GET /identity/v3/role_assignments?user.id=8745ceb73cfa4825acf3bd38d757659d&scope.project.id=eb49414f27cc49e6bc500f4dab137c38&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:09.251507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e4fd-0d81-428e-89ee-06b14dc05385 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:09.252085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e4fd-0d81-428e-89ee-06b14dc05385 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:09.252085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e4fd-0d81-428e-89ee-06b14dc05385 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:09.253384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 568/1147] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:09.258946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:09.259379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:09.259471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:09.279536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:09.280731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:09.361664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 580/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:09.632203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614e4fd-0d81-428e-89ee-06b14dc05385 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:09.702088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:09.755656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614e4fd-0d81-428e-89ee-06b14dc05385 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:09.790564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:09.791216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:09.791492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:09.791760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:09.802710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:09.807461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:09.808524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:09.809749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 569/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:09.816915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:09.884867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:09.929128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523281f0-8aef-4c6f-9ddf-9d539cd2fed6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:09.967566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:09.968176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:09.968452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:09.968723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:09.969379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:09.969679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:09.971065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:09.979559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 581/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:09.988163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:10.096642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18b2d5-2a9c-4a25-8d6d-c55be44a8cd4 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:10.119357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:10.132075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:10.132692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:10.132963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:10.133227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] PATH_INFO: `/v3/users/8745ceb73cfa4825acf3bd38d757659d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:10.134381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] RBAC: Authorizing `identity:get_user(user_id=8745ceb73cfa4825acf3bd38d757659d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:10.135902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:10.137392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 570/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:09 2022] GET /identity/v3/users/8745ceb73cfa4825acf3bd38d757659d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:10.143569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:10.247224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee80f9-5a9a-4c97-97fc-c5a7d45009fc None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:10.269272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:10.284316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:10.284954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:10.285228 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:10.285504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] PATH_INFO: `/v3/projects/eb49414f27cc49e6bc500f4dab137c38` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:10.286662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] RBAC: Authorizing `identity:get_project(project_id=eb49414f27cc49e6bc500f4dab137c38)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:10.288135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f96ad6-87a8-4a6d-9f63-bc4ace5ff041 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:10.289783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 582/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:10 2022] GET /identity/v3/projects/eb49414f27cc49e6bc500f4dab137c38 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:10.296221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:10.403607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f02a-b753-4f96-bc9a-3c27157a5663 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:10.440963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:10.441599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:10.441873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:10.442148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] PATH_INFO: `/v3/projects/eb49414f27cc49e6bc500f4dab137c38/users/8745ceb73cfa4825acf3bd38d757659d/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:10.444284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] RBAC: Authorizing `identity:create_grant(project_id=eb49414f27cc49e6bc500f4dab137c38, user_id=8745ceb73cfa4825acf3bd38d757659d, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:10.445721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04a620-26fd-44d5-95c5-87eccbc702ce None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:10.461817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 571/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:38:10 2022] PUT /identity/v3/projects/eb49414f27cc49e6bc500f4dab137c38/users/8745ceb73cfa4825acf3bd38d757659d/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:38:12.309100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6f47-dba9-44df-b48d-def97c152807 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:12.309618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6f47-dba9-44df-b48d-def97c152807 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:12.309618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6f47-dba9-44df-b48d-def97c152807 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:12.311121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 583/1154] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:12.317393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:12.318140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:12.318320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:12.346358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:12.347587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:12.399682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:12.427753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 572/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:12.693974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b6f47-dba9-44df-b48d-def97c152807 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:12.813994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b6f47-dba9-44df-b48d-def97c152807 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:12.852777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:12.853761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:12.854036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:12.854301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:12.865376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:12.870190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:12.871262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:38:12.872281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 584/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:12.878136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:12.996273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad9ead-a75b-45bc-9f3d-a56bc58f2271 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:13.031569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:13.032162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:13.032445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:13.032716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:13.033383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:13.033683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:13.035105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:13.043244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 573/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:13.049778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:13.102897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:13.154135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27d268-e4b7-4b33-9801-874720b12459 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:13.189210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:13.189821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:13.190099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:13.190364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] PATH_INFO: `/v3/users/8745ceb73cfa4825acf3bd38d757659d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:13.191588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] RBAC: Authorizing `identity:get_user(user_id=8745ceb73cfa4825acf3bd38d757659d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:13.193015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:13.194569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 585/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:13 2022] GET /identity/v3/users/8745ceb73cfa4825acf3bd38d757659d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:13.202922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:13.247523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:13.304697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8d40a-a5b5-43d8-ad7a-56fb7e15f1d0 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:13.338828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:13.339463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:13.339739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:13.340006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:13.405985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:13.407463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:13.448355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:13.520610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 574/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:38:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:38:13.530120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:13.645615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416efd-d889-48e0-a7f3-a41c354bcfef None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:13.661583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:13.680343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:13.680954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:13.681234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:13.681487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] PATH_INFO: `/v3/projects/eb49414f27cc49e6bc500f4dab137c38` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:13.682607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] RBAC: Authorizing `identity:get_project(project_id=eb49414f27cc49e6bc500f4dab137c38)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:13.684070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89142455-0f3d-4e35-9eb7-88ba829e3a4b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:13.685764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 586/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:13 2022] GET /identity/v3/projects/eb49414f27cc49e6bc500f4dab137c38 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:13.695743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:13.803120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d79a8-9b11-4fd9-bab1-d09f92aa133c None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:13.849140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:13.849739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:13.850018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:13.850295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:13.851045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=eb49414f27cc49e6bc500f4dab137c38, user.id=8745ceb73cfa4825acf3bd38d757659d) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:13.851350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:13.852800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f1ebe-03f1-48e2-8a9f-1d22a995ad36 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:13.859131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 575/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:38:13 2022] GET /identity/v3/role_assignments?user.id=8745ceb73cfa4825acf3bd38d757659d&scope.project.id=eb49414f27cc49e6bc500f4dab137c38&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:15.751411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844de03-acf3-430f-ba22-9958a6ca1f71 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:15.752243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844de03-acf3-430f-ba22-9958a6ca1f71 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:15.752243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844de03-acf3-430f-ba22-9958a6ca1f71 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:15.754442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 587/1162] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:15.760597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:15.761037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:15.761162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:15.776173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:15.782493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:15.783716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:15.850948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 576/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:16.118218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844de03-acf3-430f-ba22-9958a6ca1f71 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:16.227434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844de03-acf3-430f-ba22-9958a6ca1f71 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:16.265655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:16.266266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:16.266546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:16.266829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:16.318769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:16.335544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:16.337002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:16.441042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 588/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:38:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:38:16.447322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:16.558226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bb8b8-e409-4f95-ae07-b9d565fbbe62 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:16.593736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:16.594390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:16.594672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:16.594970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:16.597872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:16.602918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:16.605058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-02a68d0b-e155-49f9-9536-ecdeb4bdd34d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:16.617182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:16.618342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:16.619761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 577/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:38:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:16.626243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:16.715348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:16.731890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420d434-063a-46c9-81d7-bcee8e6ff67c None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:16.766335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:16.766984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:16.767216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:16.767488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:16.768112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:16.768410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:16.769754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcbf2c-e7a3-44f5-bb2b-9d0876fc8bc5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:16.776830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 589/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Sep 3 03:38:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:18.634397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199635-ad31-4f87-9798-4988b0d3a39c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:18.635311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199635-ad31-4f87-9798-4988b0d3a39c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:18.635311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199635-ad31-4f87-9798-4988b0d3a39c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:18.637391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 578/1167] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:18.642952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aab29-1635-4b95-9cb3-21b720aa00cc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:18.643386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aab29-1635-4b95-9cb3-21b720aa00cc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:18.643488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aab29-1635-4b95-9cb3-21b720aa00cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:18.663669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1e3aab29-1635-4b95-9cb3-21b720aa00cc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:18.664990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1e3aab29-1635-4b95-9cb3-21b720aa00cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:18.741550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 590/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:19.012298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87199635-ad31-4f87-9798-4988b0d3a39c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:19.120404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87199635-ad31-4f87-9798-4988b0d3a39c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:19.155091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:19.155672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:19.155941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:19.156204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:19.156881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:19.158249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1adba78-6106-4bf5-a895-eec4c51a0f72 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:19.177903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 579/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Sat Sep 3 03:38:19 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:38:21.087368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9514d6-06e0-4d6b-9e5e-94841f198f88 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:21.087946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9514d6-06e0-4d6b-9e5e-94841f198f88 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:21.087946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9514d6-06e0-4d6b-9e5e-94841f198f88 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:21.089998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 591/1170] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:21.096092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:21.096537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:21.096647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:21.115784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:21.116979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:21.173647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:21.194465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 580/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:38:21.464107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9514d6-06e0-4d6b-9e5e-94841f198f88 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:21.558391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:21.581667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9514d6-06e0-4d6b-9e5e-94841f198f88 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:21.621244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:21.621849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:21.622124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:21.622395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] PATH_INFO: `/v3/domains/da0b7de6e6d448359e480df3350f439f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:21.633798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=da0b7de6e6d448359e480df3350f439f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:21.635260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0313c-b83b-4958-b48a-6068605037bb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:21.636928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 592/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Sat Sep 3 03:38:21 2022] GET /identity/v3/domains/da0b7de6e6d448359e480df3350f439f => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:21.645364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:21.748552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485728b1-5052-4529-b05c-37a23396fa26 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:21.783579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:21.784182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:21.784462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:21.784729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:21.785379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:21.786851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4b6f-93b1-491a-8963-56ab37ae3cb4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:21.816841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 581/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:38:21 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:38:23.723554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afb49-b6a2-468f-ad2d-4759be259dfe None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:23.724387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afb49-b6a2-468f-ad2d-4759be259dfe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:23.724387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afb49-b6a2-468f-ad2d-4759be259dfe None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:23.726662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 593/1174] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:23.733495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:23.733971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:23.734066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:23.763435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:23.764623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:23.795446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:23.837722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 582/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:24.103880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20afb49-b6a2-468f-ad2d-4759be259dfe None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:24.216147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20afb49-b6a2-468f-ad2d-4759be259dfe None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:24.257536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:24.258168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:24.258450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:24.258718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] PATH_INFO: `/v3/domains/da0b7de6e6d448359e480df3350f439f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:24.259916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da0b7de6e6d448359e480df3350f439f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:24.261346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477601d6-a71b-4df3-b77d-3c76b6065523 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:24.263014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 594/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Sat Sep 3 03:38:24 2022] GET /identity/v3/domains/da0b7de6e6d448359e480df3350f439f => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:24.269415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:24.379194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ccea2-0b3c-4079-84b9-57dbe91e8475 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:24.414789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:24.415423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:24.415699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:24.415967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:24.416618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:24.418085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e9dac-106f-414c-8093-3a37b9b36063 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:24.447767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 583/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:38:24 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:38:26.363604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808529-a2b5-4928-a520-d62122ca3fa6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:26.364401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808529-a2b5-4928-a520-d62122ca3fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:26.364401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808529-a2b5-4928-a520-d62122ca3fa6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:26.366514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 595/1178] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:26.371922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:26.372412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:26.372504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:26.392837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:26.394013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:26.413856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:26.467508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 584/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:26.745808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808529-a2b5-4928-a520-d62122ca3fa6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:26.760834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:26.865833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808529-a2b5-4928-a520-d62122ca3fa6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:26.900857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:26.901474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:26.901750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:26.902018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:26.913797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:26.918567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:26.920548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:38:26.922394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 596/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:26.927352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:27.030435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6e22-2df5-4118-bb34-d3290609b17a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:27.064762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:27.065357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:27.065635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:27.065911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:27.066563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:27.066846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:27.068275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:27.076332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 585/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:27.085529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:27.161294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:27.197089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc784c8-7b49-4938-a4ee-ff8ae39a61a9 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:27.232516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:27.233129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:27.233403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:27.233672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] PATH_INFO: `/v3/users/c04ffddb683844bcb943f4e2d33087ea` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:27.247556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] RBAC: Authorizing `identity:get_user(user_id=c04ffddb683844bcb943f4e2d33087ea)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:27.248988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:27.250628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 597/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:27 2022] GET /identity/v3/users/c04ffddb683844bcb943f4e2d33087ea => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:27.256757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:27.276971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:27.363216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b356d7-9d6a-4934-81bc-6a3999da4960 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:27.397638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:27.398230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:27.398494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:27.398753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:27.466038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:27.467509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:27.484577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:27.567006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 586/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:38:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:38:27.573252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:27.681017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caa8cf-81b2-4676-91b9-13f1227fda2f None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:27.707136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:27.720051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:27.720674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:27.720952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:27.721222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] PATH_INFO: `/v3/projects/c2cbee855b7e498fab94d84d8eb8d315` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:27.722459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] RBAC: Authorizing `identity:get_project(project_id=c2cbee855b7e498fab94d84d8eb8d315)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:27.723961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59240-6c28-4989-a15e-d34931f1b966 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:27.725618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 598/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:27 2022] GET /identity/v3/projects/c2cbee855b7e498fab94d84d8eb8d315 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:27.734245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:27.845455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddac8f-09b3-4341-b33a-9d9bd52f3298 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:27.862572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:27.881740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:27.882332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:27.882608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:27.882907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:27.883636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=c2cbee855b7e498fab94d84d8eb8d315, user.id=c04ffddb683844bcb943f4e2d33087ea) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:27.883945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:27.885401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9b5e-bf61-4323-bcf8-c45b0da031f4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:27.890849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 587/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:38:27 2022] GET /identity/v3/role_assignments?user.id=c04ffddb683844bcb943f4e2d33087ea&scope.project.id=c2cbee855b7e498fab94d84d8eb8d315&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:29.749220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0051e6-0bc7-4fb5-b81b-52a22f325920 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:29.750111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0051e6-0bc7-4fb5-b81b-52a22f325920 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:29.750111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0051e6-0bc7-4fb5-b81b-52a22f325920 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:29.752314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 599/1186] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:29.758473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:29.759275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:29.759436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:29.781437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:29.782623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:29.854918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 588/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:30.133905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0051e6-0bc7-4fb5-b81b-52a22f325920 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:30.246106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0051e6-0bc7-4fb5-b81b-52a22f325920 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:30.281005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:30.281630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:30.281903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:30.282167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:30.292629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:30.297344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:30.298464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:38:30.299403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 600/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:30.304670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:30.408603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb482529-9ece-4dd9-b305-56910b79b8fc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:30.419061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:30.444950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:30.445546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:30.445821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:30.446089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:30.446752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:30.447085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:30.448449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:30.456530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 589/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:30.462382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:30.518990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:30.567709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcceaf-32fc-4ee0-9b6f-fda5c74447e2 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:30.602091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:30.602700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:30.603320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:30.603587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] PATH_INFO: `/v3/users/c04ffddb683844bcb943f4e2d33087ea` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:30.604826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] RBAC: Authorizing `identity:get_user(user_id=c04ffddb683844bcb943f4e2d33087ea)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:30.606347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:30.608270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 601/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:30 2022] GET /identity/v3/users/c04ffddb683844bcb943f4e2d33087ea => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:30.614307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:30.722573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccb553-bd68-4bad-b691-d13876998ed8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:30.758432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:30.759087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:30.759749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:30.759749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] PATH_INFO: `/v3/projects/c2cbee855b7e498fab94d84d8eb8d315` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:30.760946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=c2cbee855b7e498fab94d84d8eb8d315)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:30.762470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c67ee-f1cd-475a-9edc-a693bb23a6a4 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:30.764129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 590/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:30 2022] GET /identity/v3/projects/c2cbee855b7e498fab94d84d8eb8d315 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:30.773284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:30.879558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529fac-78fd-4898-a3fe-96e3aca9f024 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:30.902633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:30.918840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:30.919510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:30.919789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:30.920059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] PATH_INFO: `/v3/projects/c2cbee855b7e498fab94d84d8eb8d315/users/c04ffddb683844bcb943f4e2d33087ea/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:30.922176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2cbee855b7e498fab94d84d8eb8d315, user_id=c04ffddb683844bcb943f4e2d33087ea, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:30.923630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272404c-368c-4556-a7fd-f4370201dec8 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:30.939215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 602/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:38:30 2022] PUT /identity/v3/projects/c2cbee855b7e498fab94d84d8eb8d315/users/c04ffddb683844bcb943f4e2d33087ea/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:38:32.800303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019675-16b3-446c-967f-201bb0610fde None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:32.801182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019675-16b3-446c-967f-201bb0610fde None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:32.801182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019675-16b3-446c-967f-201bb0610fde None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:32.803475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 591/1193] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:38:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:38:32.808831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415351f-be26-43ad-8736-d8d42354887c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:32.809277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415351f-be26-43ad-8736-d8d42354887c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:32.809375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415351f-be26-43ad-8736-d8d42354887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:32.828996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c415351f-be26-43ad-8736-d8d42354887c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:38:32.830174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c415351f-be26-43ad-8736-d8d42354887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:38:32.902211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 603/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:38:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:38:33.169945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019675-16b3-446c-967f-201bb0610fde None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:33.225471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:33.290105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019675-16b3-446c-967f-201bb0610fde None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:33.325248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:33.325862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:33.326139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:33.326413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:33.343710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:38:33.348454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:38:33.349587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:38:33.350738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:38:33.354846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 592/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:38:33.356051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415351f-be26-43ad-8736-d8d42354887c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:33.458271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415351f-be26-43ad-8736-d8d42354887c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:33.468884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:33.500738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:33.501338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:33.501619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:33.501886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:33.502545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:33.502838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:33.504245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:33.513417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 604/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:38:33.522302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:33.628550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:33.631901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f13c-e2d4-4805-b077-3bfca4e1259a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:33.667436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:33.668083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:33.668301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:33.668810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] PATH_INFO: `/v3/users/c04ffddb683844bcb943f4e2d33087ea` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:33.670465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] RBAC: Authorizing `identity:get_user(user_id=c04ffddb683844bcb943f4e2d33087ea)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:33.672693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:33.675167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 593/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Sep 3 03:38:33 2022] GET /identity/v3/users/c04ffddb683844bcb943f4e2d33087ea => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:33.684320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:33.791180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110abd5b-103b-4982-9779-0498ad2b45e0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:33.825191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:33.825790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:33.826044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:33.826309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:33.867584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:33.896380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:33.897808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:33.996788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 605/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:38:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:38:34.006365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:34.115956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26597-1eee-467b-ab96-57ee52fa035f None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:34.150847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:34.151496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:34.151775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:34.152049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] PATH_INFO: `/v3/projects/c2cbee855b7e498fab94d84d8eb8d315` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:34.153331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] RBAC: Authorizing `identity:get_project(project_id=c2cbee855b7e498fab94d84d8eb8d315)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:34.154769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738f57c-f95a-4433-8a18-8c48771985e9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:34.156411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 594/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Sep 3 03:38:34 2022] GET /identity/v3/projects/c2cbee855b7e498fab94d84d8eb8d315 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:34.166012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:38:34.236597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:34.270029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5d1b-ecca-44a7-bb30-025b2c3f9a9d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:38:34.304877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:38:34.305463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:38:34.305737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:38:34.306003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:38:34.306715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=c2cbee855b7e498fab94d84d8eb8d315, user.id=c04ffddb683844bcb943f4e2d33087ea) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:38:34.307045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:38:34.308470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb845c1a-e4d6-4b42-9dda-81720caa35a7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:38:34.314286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 606/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:38:34 2022] GET /identity/v3/role_assignments?user.id=c04ffddb683844bcb943f4e2d33087ea&scope.project.id=c2cbee855b7e498fab94d84d8eb8d315&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:38:44.145647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=82 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:38:44.311653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=32 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:29.398045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb78e3-766f-4260-8ceb-b081b8229f38 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:29.398626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb78e3-766f-4260-8ceb-b081b8229f38 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:29.398626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb78e3-766f-4260-8ceb-b081b8229f38 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:29.400206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 595/1201] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:29.405531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:29.406068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:29.406068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:29.425169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:29.426593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:29.510940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 607/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 03:39:29.789646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb78e3-766f-4260-8ceb-b081b8229f38 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:29.922884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb78e3-766f-4260-8ceb-b081b8229f38 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:29.967791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:29.968490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:29.968766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:29.969051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:29.984024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:29.985672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:30.007183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:30.008509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:30.014470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:30.014470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:30.014470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:30.014470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 596/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:30.016706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:30.158496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d9c08-44f1-4868-a82f-0b03fee0fa71 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:30.196195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:30.207360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:30.208124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:30.208403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:30.208659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:30.209305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:30.209671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:30.211131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5792cb-95f7-4e9c-8f45-9955b76485c4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:30.223778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 608/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:30.237939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:30.307006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:30.366311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d0f73-a709-4ed7-980a-f2236c218bc2 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:30.407677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:30.408359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:30.408646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:30.408755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:30.409387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:30.410895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19da192-4fe5-42e4-a89a-0a82506b8205 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:30.438852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 597/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Sep 3 03:39:30 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:39:32.474835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235684c-e122-4816-9608-6f79360c82d8 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:32.474835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235684c-e122-4816-9608-6f79360c82d8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:32.474835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235684c-e122-4816-9608-6f79360c82d8 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:32.475526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 609/1206] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:32.480899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:32.481316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:32.481451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:32.502748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:32.504086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:32.581968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 598/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:32.866465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235684c-e122-4816-9608-6f79360c82d8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:32.989518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235684c-e122-4816-9608-6f79360c82d8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.027758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.028426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.028697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.028972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.042038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:33.049069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:33.050093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:33.051139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:33.054982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 610/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:33.056866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:33.111411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:33.177296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d144b34-34fa-48d8-a090-fea47464376d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.212661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.213283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.213565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.213835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.214505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:33.214824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:33.216218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:33.224563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 599/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:33.231380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:33.293128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:33.348850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f437-aa42-4045-a9a4-5a7ca74a5456 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.388268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.388902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.389984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.389984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.401020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:33.402485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:33.418774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:33.419847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:33.420797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:33.420797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:33.420797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:33.420797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 611/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:33.426931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:33.549756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:33.553447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0becb-fb41-498f-925d-82195c3ce666 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.600216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.602022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.602022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.602218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.602962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:33.603334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:33.604881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:33.624434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 600/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:33.626243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:33.737174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37440-8447-485e-8243-d3afd3e5f0a4 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.738736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:33.774440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.775332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.775623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.775889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.784284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:33.785745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:33.797452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:33.798673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:33.800014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:33.802947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 612/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:33.804753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:33.916846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4a98-906e-4d22-9600-24acfc90712b None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:33.953708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:33.954359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:33.954695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:33.955006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:33.955835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:33.956110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:33.957806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:33.976226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 601/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:33.982867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:34.102023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c020f-1f8d-436d-a050-a89fc8d34278 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:34.138375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:34.139233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:34.139584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:34.139966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:34.170491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:34.213729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:34.215224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:34.322909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 613/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:39:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 03:39:34.328900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:34.349578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:34.441943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0211b0-3fef-43cb-85b1-2a85b6038606 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:34.481306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:34.482004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:34.482308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:34.482606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:34.493867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:34.495553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:34.511961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:34.513283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:34.514591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:34.514591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:34.514591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:34.518677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 602/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:34.520034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:34.589535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:34.638449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4f1d6-f017-466f-8eaf-5060a2373deb None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:34.674339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:34.674950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:34.675152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:34.675396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:34.676043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:34.676336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:34.677706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:34.690145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 614/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:34.695807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:34.774645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:34.807264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151477fd-1344-42c1-afa2-3412300328e8 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:34.842896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:34.842896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:34.843241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:34.843751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:34.853117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:34.854877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:34.869092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:34.870261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:34.871517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:34.872846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 603/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:34.878883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:34.991189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2fb94-0ad3-43b9-96cc-0db62a394446 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:35.016438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:35.031149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:35.031807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:35.032139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:35.032437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:35.033257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:35.033610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:35.035261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7482e-8090-4068-bedf-b4175e0cdf12 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:35.049695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 615/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:35.055589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:35.161648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91743887-0aee-4dd6-b6a6-fd3605d84976 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:35.191923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:35.198232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:35.198867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:35.199151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:35.199424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:35.200161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=15badc5a92d745b0896aed0f7c582e73) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:35.200471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:35.201922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0269820-27e0-4b52-ab50-f5ee0ae804b6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:35.208944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 604/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:39:35 2022] GET /identity/v3/role_assignments?user.id=15badc5a92d745b0896aed0f7c582e73&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:37.099039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60690-676b-4be0-b2bb-a7290f790c4a None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:37.099671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60690-676b-4be0-b2bb-a7290f790c4a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:37.099671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60690-676b-4be0-b2bb-a7290f790c4a None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:37.101028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 616/1220] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:37.107549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:37.108058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:37.108288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:37.128987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:37.130184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:37.204168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 605/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:37.484348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e60690-676b-4be0-b2bb-a7290f790c4a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:37.593976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e60690-676b-4be0-b2bb-a7290f790c4a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:37.634499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:37.635192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:37.635457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:37.635719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:37.646400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:37.651152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:37.652288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:37.653283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 617/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:37.658558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:37.764340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee6772-c50c-42c6-a43d-ba51b9d14970 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:37.788583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:37.800347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:37.800963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:37.801239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:37.801513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:37.802173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:37.802476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:37.803889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:37.812688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 606/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:37.818466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:37.864271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:37.922421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45e2f1-0799-4470-a62c-db703d5524a3 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:37.959251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:37.959976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:37.960254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:37.960557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:37.971343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:37.972959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:37.988154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:37.989317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:37.990272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:37.990272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:37.990272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:37.990272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 618/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:37.995324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.102390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12dc45-5edf-40df-948d-ef0e0df127b9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:38.142135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:38.142835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:38.143054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:38.143412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:38.144238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:38.144686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:38.147094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:38.160038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 607/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:38.165626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.270754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:38.278314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95a253-4ecf-495b-a5d1-c4a62cfa6d07 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:38.314245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:38.314910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:38.315190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:38.315461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:38.323468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:38.324943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:38.336366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:38.337693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:38.339103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 619/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:38.344037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.393540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:38.453956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf498678-f734-4a10-93de-09502dda6600 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:38.488580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:38.489195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:38.489470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:38.489740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:38.490466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:38.490771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:38.492279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:38.509739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 608/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:38.515533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.619930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91a17-e544-40f3-accd-11a193bd4e05 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:38.655754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:38.656314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:38.656588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:38.656880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:38.668853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:38.670406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:38.686926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:38.688169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:38.689271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:38.689271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:38.689271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:38.689271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 620/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:38.694789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.816345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c104-a0e1-4752-ab78-ec2321b2b8ea None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:38.819134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:38.859519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:38.860234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:38.860490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:38.860771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:38.861418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:38.861794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:38.863310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:38.876105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 609/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:38.882390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:38.892622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:38.992843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5700f-aeb3-4b61-ab6b-34e9d93f89bd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:39.028841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:39.029599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:39.029917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:39.030240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:39.037685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:39.039364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:39.051708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:39.052815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:39.054039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 621/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:39:39.059751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:39.170603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb1ba-70e0-4454-bb6e-f9f6ff8e6882 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:39.210410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:39.211406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:39.211683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:39.212001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:39.212801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:39.213101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:39.214769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ccaf4-90ea-4ec5-881b-844433fe9372 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:39.228370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 610/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:39.234701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:39.309904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:39.350994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d26cf-3b10-4daa-b402-5b713d9bb86e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:39.387442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:39.388099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:39.388374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:39.388665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/15badc5a92d745b0896aed0f7c582e73/roles/ec4e0c4104c246a8a405beb17eca8bb5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:39.404205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=15badc5a92d745b0896aed0f7c582e73, role_id=ec4e0c4104c246a8a405beb17eca8bb5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:39.405754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5ef66-c8e0-4f06-9b36-3505604d5862 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:39.419939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 622/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:39:39 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/15badc5a92d745b0896aed0f7c582e73/roles/ec4e0c4104c246a8a405beb17eca8bb5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:39:41.383673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e9c-8e81-4fb0-a793-7aa9cef547e6 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:41.384298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e9c-8e81-4fb0-a793-7aa9cef547e6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:41.384298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e9c-8e81-4fb0-a793-7aa9cef547e6 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:41.385680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 611/1233] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:39:41.391430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:41.391903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:41.392083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:41.417676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:41.419000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:41.487262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 623/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:41.748714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d0e9c-8e81-4fb0-a793-7aa9cef547e6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:41.775346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:41.859355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d0e9c-8e81-4fb0-a793-7aa9cef547e6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:41.894671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:41.895341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:41.895616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:41.895883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:41.907577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:41.912460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:41.913399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:41.914274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 612/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Sep 3 03:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:41.920049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:41.971007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:42.029109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325841c9-9f16-43b6-8dcb-b4792b5a53ee None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.062934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.063547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.063827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.064099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:42.064766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] RBAC: Adding query filter params (name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:42.065067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:42.066464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:42.075164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 624/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Sep 3 03:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:42.081483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:42.172597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:42.188362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa64a-605f-417d-81c3-fa725e0f01a1 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.225908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.226882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.227162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.227434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:42.238583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:42.240102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:42.254935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:42.256283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:42.257433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:42.257433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:42.257433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:42.257433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 613/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:42.262387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:42.364102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bedea-6dba-4d71-994d-6cedf0450f43 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.374427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:42.401868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.402499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.402745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.403713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:42.403713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:42.403960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:42.405326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:42.419795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 625/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:42.425678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:42.543277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321e6cd-3e6f-4b5f-8d18-933c16cb8458 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.579140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.579829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.580177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.580487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:42.583704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:42.590266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:42.591764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:42.604032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:42.605019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:42.606102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 614/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:42.611182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:42.714561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b24671-4cfb-48a4-a590-32fb0a699980 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.751490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.752259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.752557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.752825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:42.753810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:42.754155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:42.755860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:42.775870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 626/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:42.782741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:42.898431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ba7-8811-4161-b8fe-99d2611954b9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:42.935069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:42.935742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:42.936072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:42.936391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:43.007238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:43.015183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:43.016693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:43.132726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 615/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:39:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:39:43.141212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:43.174653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:43.259862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540562-569b-494b-909c-c5e6df01a5c5 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:43.301932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:43.302556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:43.302852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:43.303119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:43.315533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:43.317518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:43.335104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:43.336525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:43.337776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:43.337776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:43.337776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:43.337776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 627/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:43.342791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:43.444190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:43.450672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45028dad-9d5b-4a25-ae86-bdaee68da257 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:43.491098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:43.491838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:43.492182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:43.492456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:43.493263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:43.493695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:43.495227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:43.507937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 616/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:43.514425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:43.599752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:43.623934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2bc6-d069-41ff-b9d7-c90423157ebf None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:43.659832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:43.660545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:43.660873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:43.661193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:43.668993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:43.670941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:43.685501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:43.686949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:43.688154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 628/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:43.693623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:43.808559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4283f39-fecc-44f7-bba7-f22c1dee28df None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:43.849059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:43.849657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:43.849932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:43.850294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:43.851114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:43.851712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:43.853405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1b90a-63dd-4cac-8012-b568031f5af6 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:43.868999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 617/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:43.875886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:43.984917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b55f0-ab43-4afb-b9fc-7d6041e5fe0e None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:44.022067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:44.022679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:44.022987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:44.023260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:44.023986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] RBAC: Adding query filter params (role.id=ec4e0c4104c246a8a405beb17eca8bb5, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=15badc5a92d745b0896aed0f7c582e73) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:44.024290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:44.025780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9093af8-6e45-4da8-8914-b3b779e75f02 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:44.031843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 629/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:39:43 2022] GET /identity/v3/role_assignments?user.id=15badc5a92d745b0896aed0f7c582e73&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=ec4e0c4104c246a8a405beb17eca8bb5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:45.999112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c672-dc0f-4514-a4cc-bb3f5800a7cf None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:45.999682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c672-dc0f-4514-a4cc-bb3f5800a7cf None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:45.999682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c672-dc0f-4514-a4cc-bb3f5800a7cf None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:46.001166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 618/1247] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:46.006208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:46.006658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:46.006870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:46.013952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:46.029642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:46.031145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:46.105642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 630/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:46.371589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2c672-dc0f-4514-a4cc-bb3f5800a7cf None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:46.392868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:46.496015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2c672-dc0f-4514-a4cc-bb3f5800a7cf None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:46.539255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:46.539853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:46.540127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:46.540395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:46.556383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:46.562355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:46.563741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:46.564782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 619/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:46.573133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:46.687072 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e9e62-79e6-4eeb-abda-01055770394e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:46.726525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:46.732874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:46.733582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:46.733943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:46.734198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:46.734950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:46.735344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:46.736889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:46.746451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 631/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:46.752987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:46.860818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:46.890845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7a65-2242-4351-91ed-a8669f79e896 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:46.931848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:46.932697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:46.933103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:46.933374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:46.946937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:46.948429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:46.965228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:46.966832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:46.968195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:46.968195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:46.968195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:46.968195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 620/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:46.972799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:47.092801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde84d4a-7469-4509-b741-f71cfffdcdb0 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:47.134369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:47.135628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:47.135628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:47.135628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:47.136345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:47.136637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:47.138219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:47.151564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 632/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:47.158135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:47.276046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c44c0-a943-40cd-bb70-471f48d66c70 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:47.305975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:47.314638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:47.315304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:47.315583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:47.315853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:47.323708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:47.325585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:47.338517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:47.339839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:47.341173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:47.344840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 621/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:39:47.346058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:47.420927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:47.474121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b5327-ba50-43c5-b3da-740cad6c1c77 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:47.512926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:47.513570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:47.513887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:47.514212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:47.515087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:47.515392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:47.516966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:47.538021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 633/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:47.544321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:47.662532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc90883-883d-44ef-8a76-44f886fa6550 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:47.701418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:47.702102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:47.702391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:47.702717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:47.774998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:47.790892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:47.792621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:47.905728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 622/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:39:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:39:47.912092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:48.030420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b802-d6a8-43fe-a467-a0995c4972ee None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:48.044649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:48.067527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:48.068226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:48.068550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:48.068836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:48.081240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:48.083023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:48.097667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:48.099400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:48.100643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:48.100643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:48.100643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:48.100643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 634/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:39:48.105334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:48.209806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:48.224236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc76d-442a-4524-9499-9a4c0165f86a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:48.273100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:48.273819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:48.274175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:48.274519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:48.275286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:48.276566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:48.277962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:48.289970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 623/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:48.296655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:48.405009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c32c6-7f2f-49d2-ab3d-3e46d46def53 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:48.444118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:48.444750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:48.445026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:48.445299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:48.452832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:48.454445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:48.467049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:48.468131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:48.469189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 635/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:48.474553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:48.589097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0859ea-0685-4f45-89c7-feffb5191d20 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:48.633230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:48.633959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:48.634303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:48.634640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:48.635539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:48.635897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:48.637627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6a0d7-d019-46ce-8899-2ad5f425f501 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:48.651054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 624/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:48.658074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:48.687185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:48.783459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6546e60-1d22-4ddb-b3e7-303073826504 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:48.819246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:48.819854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:48.820146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:48.820423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:48.821142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=15badc5a92d745b0896aed0f7c582e73) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:48.821536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:48.822982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb61d6-618d-410d-9b78-495479c41074 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:48.843186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 636/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:39:48 2022] GET /identity/v3/role_assignments?user.id=15badc5a92d745b0896aed0f7c582e73&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:50.752003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6d63-7dda-44d1-a286-3264169ebfb3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:50.752852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6d63-7dda-44d1-a286-3264169ebfb3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:50.752852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6d63-7dda-44d1-a286-3264169ebfb3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:50.755067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 625/1261] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:50.760303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:50.760724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:50.760811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:50.780208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:50.781361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:50.849240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 637/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:51.112564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab6d63-7dda-44d1-a286-3264169ebfb3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:51.130539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:51.238997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab6d63-7dda-44d1-a286-3264169ebfb3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:51.274505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:51.275200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:51.275470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:51.275733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:51.286194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:51.291065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:51.292175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:51.293072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 626/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:51.300819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:51.326912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:51.416550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969737a-36b1-49a9-aa75-d75b3a039849 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:51.451526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:51.452116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:51.452388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:51.452655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:51.453312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:51.453605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:51.455000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:51.462948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 638/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:51.472336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:51.548810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:51.582260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6390dd-c5a9-4a38-8293-71672224a8c5 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:51.617430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:51.618032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:51.618346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:51.618619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:51.628795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:51.630233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:51.644701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:51.645780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:51.646897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:51.646897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:51.646897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:51.650461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 627/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:51.652238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:51.755627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:51.774796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554888-fcb7-4ca1-b8fb-e453124743dd None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:51.812485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:51.813085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:51.813358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:51.813625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:51.814262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:51.814562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:51.815954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:51.828142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 639/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:51.836903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:51.939645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac6cc2-98b7-429c-b2ee-f83f22357365 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:51.959317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:51.976760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:51.977370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:51.977693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:51.977963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:51.986082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:51.987545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:51.998541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:51.999679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:52.001106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 628/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:52.005463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.108155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe83e-26cd-428f-a0e4-7fde33aeb865 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:52.147202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:52.147801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:52.148073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:52.148342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:52.149058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:52.149354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:52.150767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:52.159679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:52.168601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 640/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:52.174970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.285481 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5ff5-b76a-42e8-82e0-8d15824e50fc None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:52.324915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:52.325543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:52.325817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:52.326083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:52.336621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:52.338185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:52.353467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:52.354667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:52.355825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:52.355825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:52.355825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:52.355825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 629/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:52.361221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.464087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ca21f-702d-458b-a15d-868cbb02bf89 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:52.499723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:52.500314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:52.500590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:52.500856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:52.501488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:52.501785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:52.503177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:52.514598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 641/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:52.520981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.556247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:52.643552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1461b4-16b0-46ed-8fd0-c7636f2df254 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:52.680752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:52.681562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:52.681877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:52.682141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:52.690061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:52.692076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:52.705559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:52.706749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:52.707914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 630/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:52.713776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.775082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:52.826386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b89121-b075-4ee9-aae7-b540208bc201 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:52.863333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:52.863925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:52.864196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:52.864464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:52.865144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:52.865447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:52.866873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979535d-5837-42de-bb22-50035f39ca45 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:52.878672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 642/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:52.885024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:52.979288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:52.990578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789413af-6817-4ab4-89e7-a57ce566b154 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:53.025673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:53.026318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:53.026590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:53.026865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] PATH_INFO: `/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/15badc5a92d745b0896aed0f7c582e73/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:53.029019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa493a790af417a97ccfb7b24b5f401, user_id=15badc5a92d745b0896aed0f7c582e73, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:53.030458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a0bc-f3e5-4085-ae98-d98b3629fcd5 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:53.046032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 631/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Sep 3 03:39:52 2022] PUT /identity/v3/projects/9aa493a790af417a97ccfb7b24b5f401/users/15badc5a92d745b0896aed0f7c582e73/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:39:54.912822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722029e-3acf-4e7f-86f4-81fee456d7f8 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:54.913670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722029e-3acf-4e7f-86f4-81fee456d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:54.913670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722029e-3acf-4e7f-86f4-81fee456d7f8 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:54.915880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 643/1274] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:54.921201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:54.921618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:54.921725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:54.942019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:54.943235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:55.011924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 632/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:55.277399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722029e-3acf-4e7f-86f4-81fee456d7f8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:55.405657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722029e-3acf-4e7f-86f4-81fee456d7f8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:55.422404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:55.439625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:55.440252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:55.440527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:55.440796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:55.451316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 03:39:55.456079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:55.457109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:55.458276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 644/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Sep 3 03:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:55.463386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:55.546196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:55.566628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae696-3b8e-4eb4-927b-1c85f4e80ff1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:55.600322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:55.600923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:55.601510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:55.601777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:55.602493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:55.602788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:55.604194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:55.612151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 633/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Sep 3 03:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:55.618031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:55.734685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691be7-6cfd-4228-817c-35f06500e5ea None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:55.770156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:55.770773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:55.771081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:55.771349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:55.781581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:55.783051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:55.797919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:55.799013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:55.799893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:55.799893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:55.799893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:55.799893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 645/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 03:39:55.805045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:55.909567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee2f4-5126-49fd-afa4-33d641fdf7ca None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:55.945725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:55.946395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:55.946697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:55.946999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:55.947641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:55.947944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:55.949317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:55.961049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 634/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:55.967179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:56.000126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:56.080646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a2912-f099-428a-b32f-fc467d86e357 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:56.115205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:56.115789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:56.116066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:56.116327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:56.123719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:56.125142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 03:39:56.136135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:56.137266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:56.139079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 646/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:56.146737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:56.151932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:56.263956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef488-86c0-48ac-bb81-bc80e829a38a None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:56.301012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa272-41cf-469e-8219-9459af438adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:56.301613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa272-41cf-469e-8219-9459af438adb None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:56.301887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa272-41cf-469e-8219-9459af438adb None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:56.302153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa272-41cf-469e-8219-9459af438adb None admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:56.302899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa272-41cf-469e-8219-9459af438adb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:56.303202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa272-41cf-469e-8219-9459af438adb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:56.304637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa272-41cf-469e-8219-9459af438adb None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:56.322869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 635/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Sep 3 03:39:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:56.334720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:56.433557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:56.446597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a676c2-1a06-4431-8df3-4f1e38b3d465 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:56.480470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:56.481062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:56.481335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:56.481601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:56.551539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:56.552962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:56.645440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:56.661693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 647/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Sep 3 03:39:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 03:39:56.668418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa272-41cf-469e-8219-9459af438adb None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:56.743367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:56.782642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa272-41cf-469e-8219-9459af438adb None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:56.818346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:56.819000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:56.819269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:56.819535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:56.829655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:56.831136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 03:39:56.845654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:56.846773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 03:39:56.847739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 03:39:56.847739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 03:39:56.847739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:56.847739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 636/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:56.853634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:56.970035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e867c-8a58-4f66-b526-6411220e236d None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:57.005529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:57.006125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:57.006396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:57.006660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:57.007315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:57.007613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:57.008994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:57.020163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 648/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Sep 3 03:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:57.027614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:57.145115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c99e-e139-4bc0-a242-9ab3e55758bf None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:57.158598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:57.182026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:57.182643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:57.182950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:57.183218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:57.189906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:57.191388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 03:39:57.203528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:57.204541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: ERROR keystone.server.flask.application Sep 03 03:39:57.206267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 637/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Sep 3 03:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:57.213831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:57.278509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:57.335390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6efec-8a58-4632-aed7-5245d89bfab7 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:57.369629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:57.370263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:57.370532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:57.370793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:57.371514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:57.371814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:57.373224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c566-7a0b-4f74-97dc-c110e6b7b1be None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:57.384981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 649/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Sep 3 03:39:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:39:57.394138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:57.499572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137fdd7-3971-4425-b514-71bacdb4aee9 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:57.534515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:57.535060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:57.535332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:57.535600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:57.536311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] RBAC: Adding query filter params (role.id=c0d6e38a7c014c9cbdbb8dc87533008a, scope.project.id=9aa493a790af417a97ccfb7b24b5f401, user.id=15badc5a92d745b0896aed0f7c582e73) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:39:57.536611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:57.538041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8e1dd-a866-4b4f-8c88-fbf8cf671c80 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:57.544326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 638/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Sep 3 03:39:57 2022] GET /identity/v3/role_assignments?user.id=15badc5a92d745b0896aed0f7c582e73&scope.project.id=9aa493a790af417a97ccfb7b24b5f401&role.id=c0d6e38a7c014c9cbdbb8dc87533008a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:39:59.426438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc49fe3-29af-4f85-9867-bfd834ae551e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:59.427383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc49fe3-29af-4f85-9867-bfd834ae551e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:59.427383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc49fe3-29af-4f85-9867-bfd834ae551e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:59.429489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 650/1288] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:39:59.435888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:59.436576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:59.436765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:59.459588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:39:59.460788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:39:59.492016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:59.534790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 639/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:39:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:39:59.810336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc49fe3-29af-4f85-9867-bfd834ae551e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:39:59.928750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc49fe3-29af-4f85-9867-bfd834ae551e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:39:59.939280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:39:59.964503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:39:59.965117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:39:59.965394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:39:59.965623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:39:59.966232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:39:59.967625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:39:59.976885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:39:59.978099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:39:59.979461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 651/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:39:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:39:59.984707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:00.089752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbad085-bdcf-419d-8a55-d567ec2d45b5 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:00.126230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:00.126849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:00.127124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:00.127391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:00.128043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:40:00.128342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:00.129711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd441e-826b-450f-ba7b-ab2aac4c19bf None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:00.134671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 640/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Sep 3 03:39:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:40:00.141447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:00.266894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37b3e-d002-4522-932d-15aad9c80b75 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:00.304329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:00.304928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:00.305201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:00.305467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] PATH_INFO: `/v3/services` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:00.306084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] RBAC: Adding query filter params (type=placement) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:40:00.306381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:00.307776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bda0f-a6ed-452f-a5f5-3aa6a351e9cc None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:00.312296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 652/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Sep 3 03:40:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:40:02.216871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c19cc6-37a3-4c6d-ba81-bbcbe2c1e045 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:02.217759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c19cc6-37a3-4c6d-ba81-bbcbe2c1e045 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:02.217759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c19cc6-37a3-4c6d-ba81-bbcbe2c1e045 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:02.219906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 641/1293] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:40:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:40:02.226488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdddff6-3022-401a-9e53-b9229926e84a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:02.226961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdddff6-3022-401a-9e53-b9229926e84a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:02.227105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdddff6-3022-401a-9e53-b9229926e84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:02.250593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4bdddff6-3022-401a-9e53-b9229926e84a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:40:02.252325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4bdddff6-3022-401a-9e53-b9229926e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:40:02.287613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:02.331217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 653/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:40:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:40:02.600334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c19cc6-37a3-4c6d-ba81-bbcbe2c1e045 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:02.648778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:02.739781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c19cc6-37a3-4c6d-ba81-bbcbe2c1e045 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:02.776030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:02.776630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:02.776903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:02.777169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:02.777810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:02.779215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d5ba0-0eff-47ff-87da-c621bebab46c None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:02.795246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 642/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Sep 3 03:40:02 2022] POST /identity/v3/services => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:40:04.662353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560212c-0e1c-40a2-a9dc-0c438ab6b2ac None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:04.663216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560212c-0e1c-40a2-a9dc-0c438ab6b2ac None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:04.663216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560212c-0e1c-40a2-a9dc-0c438ab6b2ac None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:04.665066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 654/1296] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:40:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:40:04.671331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:04.671766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:04.671896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:04.701006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:40:04.702203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:40:04.780142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 643/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:40:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:40:05.039872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560212c-0e1c-40a2-a9dc-0c438ab6b2ac None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:05.145467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560212c-0e1c-40a2-a9dc-0c438ab6b2ac None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:05.180042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:05.180722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:05.180990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:05.181262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:05.181824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:05.183202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:40:05.192040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:40:05.193162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:05.194015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 655/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:40:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:40:05.201536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:05.224815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:05.315922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb3377-15f9-4be3-b838-053b68c86a13 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:05.353626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:05.354237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:05.354509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:05.354777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:05.355441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:40:05.355735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:05.357109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:05.362885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 644/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Sep 3 03:40:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:40:05.369260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:05.408042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:05.486702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc594-e141-40e0-9a27-27e34befca78 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:05.521676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:05.522275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:05.522552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:05.522849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:05.523539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] RBAC: Adding query filter params (interface=public, service_id=0ebdbf8e0ffe4fae998b405acbb94d05, region_id=RegionOne) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:40:05.523875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:05.525251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9ee3a-6fbf-41ec-a82c-dd169e39851d None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:05.530379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 656/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Sep 3 03:40:05 2022] GET /identity/v3/endpoints?service_id=0ebdbf8e0ffe4fae998b405acbb94d05&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:40:05.539742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:05.622675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:05.643343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee356eea-7899-4bb6-9eb9-263a61bfd372 None admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:05.682874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:05.683668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:05.683867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:05.684395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:05.685090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:05.687468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8caff-42e0-4c77-a3aa-887384d427ca None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:05.697972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 645/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Sep 3 03:40:05 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:40:07.561346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d827d-66eb-4a5e-8fe5-66c3a12d7018 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:07.561889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d827d-66eb-4a5e-8fe5-66c3a12d7018 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:07.561889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d827d-66eb-4a5e-8fe5-66c3a12d7018 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:07.563186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 657/1302] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:40:07.569417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:07.570144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:07.570305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:07.590999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:40:07.592172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:40:07.657479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 646/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:40:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 03:40:07.927859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d827d-66eb-4a5e-8fe5-66c3a12d7018 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:08.030598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:08.043736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d827d-66eb-4a5e-8fe5-66c3a12d7018 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:08.088817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:08.089446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:08.089723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:08.089995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:08.090580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:08.091988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 03:40:08.101358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 03:40:08.102509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application with Lock( Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self._enter() Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application return self.creator() Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application created_value = creator( Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: ERROR keystone.server.flask.application Sep 03 03:40:08.103391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 658/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Sep 3 03:40:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 03:40:08.108772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:08.189651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:08.220615 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b232a-c3e8-4b3a-905a-c62e7d3d2516 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:08.254291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:08.254965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:08.255203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:08.255504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] PATH_INFO: `/v3/services` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:08.256122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:40:08.256417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:08.257800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2e6e-aaa0-4952-95b3-18b18f0a8121 None admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:08.263079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 647/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Sep 3 03:40:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:40:08.269165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:40:08.380834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1df5f-b4db-44ef-a8ae-791964f5fd16 None admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:40:08.417506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:40:08.418119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:40:08.418399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:40:08.418665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:40:08.419373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:40:08.420754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77e898-ab98-4aa9-8eac-76647713f38b None admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:40:08.444065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 659/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:40:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:40:18.261295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=30 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:40:18.439090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:12.054067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f40-4da7-4b87-8659-9472675a6ed1 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:12.055206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f40-4da7-4b87-8659-9472675a6ed1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:12.055692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f40-4da7-4b87-8659-9472675a6ed1 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:12.063014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e02d-b64c-4e17-b372-bd52d50f78fa None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:12.063874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e02d-b64c-4e17-b372-bd52d50f78fa None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:12.064367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e02d-b64c-4e17-b372-bd52d50f78fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:12.069381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 648/1307] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:12 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:12.151601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-af84e02d-b64c-4e17-b372-bd52d50f78fa None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:12.153294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-af84e02d-b64c-4e17-b372-bd52d50f78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:12.321750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 660/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:12.631126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:12.631703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:12.632146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:12.633939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 649/1309] 10.222.0.25 () {54 vars in 965 bytes} [Sat Sep 3 03:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:12.640112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603e23e-5cdc-4b36-95a9-47bdcd8680e3 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:12.640467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603e23e-5cdc-4b36-95a9-47bdcd8680e3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:12.640828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603e23e-5cdc-4b36-95a9-47bdcd8680e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:12.722184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8603e23e-5cdc-4b36-95a9-47bdcd8680e3 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:12.723676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8603e23e-5cdc-4b36-95a9-47bdcd8680e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:12.746977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:12.808001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 661/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Sat Sep 3 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:12.816867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:12.818968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:12.970974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74da94-653e-4397-b3e7-cb5c5b970dac None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:13.061732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:13.063212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:13.063767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:13.064234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:13.066144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:13.125942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:13.186548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:13.188532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0bcd-2681-4bac-bb6f-bdc2cf2d6cb4 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:13.227311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 650/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:15.429554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f7278-dfe1-4ae4-b71b-d9516e0d7661 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:15.430164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f7278-dfe1-4ae4-b71b-d9516e0d7661 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:15.430164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f7278-dfe1-4ae4-b71b-d9516e0d7661 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:15.431498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 662/1312] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:15.437406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ceb2-9388-400e-9331-cf9a9ee4e5ae None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:15.437958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ceb2-9388-400e-9331-cf9a9ee4e5ae None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:15.437958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ceb2-9388-400e-9331-cf9a9ee4e5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:15.517146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1664ceb2-9388-400e-9331-cf9a9ee4e5ae None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:15.518372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1664ceb2-9388-400e-9331-cf9a9ee4e5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:15.623218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 651/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:15.921271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:15.921548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:15.921800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:15.923196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 663/1314] 10.222.0.25 () {54 vars in 965 bytes} [Sat Sep 3 03:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:15.930305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdff79a-33af-41b3-9101-af63be8f92f0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:15.930730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdff79a-33af-41b3-9101-af63be8f92f0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:15.931208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdff79a-33af-41b3-9101-af63be8f92f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:15.947368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:16.015001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bfdff79a-33af-41b3-9101-af63be8f92f0 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:16.016121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bfdff79a-33af-41b3-9101-af63be8f92f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:16.102049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 652/1315] 10.222.0.25 () {58 vars in 1062 bytes} [Sat Sep 3 03:41:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:16.111203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:16.112734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:16.226842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd9560-6b57-4806-b8b9-9beb81476983 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:16.300713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:16.303665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:16.304321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:16.304610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:16.304885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:16.307433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:16.420933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:16.422852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7945dc-5e8f-4c7d-9e6a-f58418b0374e service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:16.445522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 664/1316] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:18.884251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:18.884809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:18.884809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:18.887259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 653/1317] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:18.896002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c862-a63f-401b-825c-0d401553b10d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:18.896400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c862-a63f-401b-825c-0d401553b10d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:18.896610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c862-a63f-401b-825c-0d401553b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:18.969870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-13d7c862-a63f-401b-825c-0d401553b10d None None] MFA Rules not processed for user `1ccc32293d63478eb9e68ea2e8c599ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:18.971131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-13d7c862-a63f-401b-825c-0d401553b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:19.024691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:19.072689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 665/1318] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:19.353482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:19.355046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:19.466139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1617e-63c7-47ed-9754-31d29185e780 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:19.470562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:19.551587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:19.552147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:19.552417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:19.552687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:19.554264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:19.669234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:19.670675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceaaa2-9dd7-401b-b9ac-b449d9dfd1fd service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:19.689246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 654/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:22.124733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:22.125279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:22.125279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:22.126550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 666/1320] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:22.133048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a1d1-4567-4192-ac89-104350d44566 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:22.133462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a1d1-4567-4192-ac89-104350d44566 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:22.133598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a1d1-4567-4192-ac89-104350d44566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:22.187069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:22.218679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8be3a1d1-4567-4192-ac89-104350d44566 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:22.220656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8be3a1d1-4567-4192-ac89-104350d44566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:22.328096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 655/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:22.610770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:22.612343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:22.730487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a84b4-3552-41b5-8ae7-5a7619e77573 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:22.825385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:22.825995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:22.826271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:22.826533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:22.828160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:22.835220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:22.951723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:22.953199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824db71-364b-4476-99da-49d4a639611b service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:22.972724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 667/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:41:25.739538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:25.740132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:25.740132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:25.741413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 656/1323] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:25.746787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7a012-ea43-4ace-a74f-0eb5e3387fd1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:25.747250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7a012-ea43-4ace-a74f-0eb5e3387fd1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:25.747347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7a012-ea43-4ace-a74f-0eb5e3387fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:25.816109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b2a7a012-ea43-4ace-a74f-0eb5e3387fd1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:25.817877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b2a7a012-ea43-4ace-a74f-0eb5e3387fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:25.877762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:25.918184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 668/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:26.200911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:26.202463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:26.311386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cf065-bf3d-4a57-a6e2-20608a5bb00a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:26.324166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:26.394954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:26.395496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:26.395773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:26.396038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:26.397623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:26.506955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:26.508389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e146-8a90-4b6d-81d3-7c6f0918bccd service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:26.527238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 657/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:28.470685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:28.471284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:28.471284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:28.472627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 669/1326] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:28.478373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63d6b-5783-42f6-b12d-23ca3d58ed24 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:28.478845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63d6b-5783-42f6-b12d-23ca3d58ed24 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:28.478971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63d6b-5783-42f6-b12d-23ca3d58ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:28.530166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:28.548993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-72d63d6b-5783-42f6-b12d-23ca3d58ed24 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:28.550226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-72d63d6b-5783-42f6-b12d-23ca3d58ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:28.651895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 658/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:28.929172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:28.930708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:29.043173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9feb7-297d-4ccc-9dff-7823fc6b05f1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:29.122039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:29.122653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:29.122953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:29.123217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:29.124816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:29.143432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:29.237489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:29.239023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d106f-195b-43d3-a95a-e03222ecd5a9 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:29.259323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 670/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:31.209405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:31.209981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:31.209981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:31.211345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 659/1329] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:31.216720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3e174-58d0-4e84-9de0-2f08d5e5dcc0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:31.217152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3e174-58d0-4e84-9de0-2f08d5e5dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:31.217276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3e174-58d0-4e84-9de0-2f08d5e5dcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:31.286242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-18f3e174-58d0-4e84-9de0-2f08d5e5dcc0 None None] MFA Rules not processed for user `1ccc32293d63478eb9e68ea2e8c599ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:31.287516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-18f3e174-58d0-4e84-9de0-2f08d5e5dcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:31.359643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:31.393756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 671/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:31.670645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:31.672305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:31.791559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291547f3-180d-41f3-952f-4faa700a7b78 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:31.816324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:31.873114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:31.873673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:31.873954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:31.874225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:31.875985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:31.994197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:31.995690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe6817-8249-40f6-ae0e-6f2844e7d777 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:32.013564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 660/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:34.153245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:34.154100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:34.154100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:34.156155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 672/1332] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:34.163549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d5008-107d-4c90-ab50-651f9a4ac666 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:34.163999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d5008-107d-4c90-ab50-651f9a4ac666 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:34.164119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d5008-107d-4c90-ab50-651f9a4ac666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:34.228866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:34.232091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-cc6d5008-107d-4c90-ab50-651f9a4ac666 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:34.233897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cc6d5008-107d-4c90-ab50-651f9a4ac666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:34.333157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 661/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:34.611356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:34.613897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:34.730587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c399-16b7-4d9d-adae-094362a375a4 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:34.815249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:34.815874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:34.816130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:34.816441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:34.818969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:34.850088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:34.947369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:34.948803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea9fb3-216e-4384-b2dc-6b4be09022c0 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:34.968766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 673/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:36.889374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:36.889962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:36.889962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:36.892111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 662/1335] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:36.902535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8ef6-f200-4617-b57c-e759bbfd65e8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:36.903010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8ef6-f200-4617-b57c-e759bbfd65e8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:36.903180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8ef6-f200-4617-b57c-e759bbfd65e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:36.980636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-06cc8ef6-f200-4617-b57c-e759bbfd65e8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:36.981835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-06cc8ef6-f200-4617-b57c-e759bbfd65e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:37.065794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:37.097639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 674/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:37.380954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:37.382487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:37.493476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5dbad-110d-4654-a1b4-9ce36b89f626 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:37.521396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:37.568352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:37.569317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:37.569740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:37.570168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:37.572855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:37.702118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:37.703582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c373f-3849-467e-807e-82f4ea53dd43 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:37.723367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 663/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:41:39.920198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:39.920752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:39.920752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:39.922110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 675/1338] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:39.928432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50351d-4199-43f3-858a-bd6fc61ebefc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:39.929146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50351d-4199-43f3-858a-bd6fc61ebefc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:39.929350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50351d-4199-43f3-858a-bd6fc61ebefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:40.002286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2a50351d-4199-43f3-858a-bd6fc61ebefc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:40.003505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2a50351d-4199-43f3-858a-bd6fc61ebefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:40.011865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:40.109253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 664/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:40.395008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:40.396560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:40.502260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b42a1b-d115-4b62-923a-8d980b58c687 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:40.575394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:40.575995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:40.576269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:40.576535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:40.578169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:40.614599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:40.692740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:40.694373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebc213-f573-4553-bcf7-b0eb336a7708 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:40.717505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 676/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:44.700211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:44.700726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:44.700954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:44.702442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 665/1341] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:44.707737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0ffa3-1715-42e5-a52f-4d393abe42fc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:44.708175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0ffa3-1715-42e5-a52f-4d393abe42fc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:44.708269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0ffa3-1715-42e5-a52f-4d393abe42fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:44.774836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8cf0ffa3-1715-42e5-a52f-4d393abe42fc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:44.776036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8cf0ffa3-1715-42e5-a52f-4d393abe42fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:44.870960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:44.886542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 677/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:45.170764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:45.172414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:45.278255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3021c-468d-42b7-8e94-97daf07f4dcf None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:45.317341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:45.353742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:45.354359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:45.354639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:45.354945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:45.356557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:45.470748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:45.472229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e9c9-03f9-435f-a04a-a3ff41022f06 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:45.491147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 666/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:48.048339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:48.048909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:48.048909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:48.050259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 678/1344] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:48.056166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10b47-c815-4f8b-8e91-6f8297a800ee None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:48.056682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10b47-c815-4f8b-8e91-6f8297a800ee None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:48.057098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10b47-c815-4f8b-8e91-6f8297a800ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:48.129460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2bd10b47-c815-4f8b-8e91-6f8297a800ee None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:48.130667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2bd10b47-c815-4f8b-8e91-6f8297a800ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:48.152044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:48.242774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 667/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:48.525086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:48.526640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:48.636297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f611f-c403-4cb0-b50e-f0c45b156012 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:48.720709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:48.721320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:48.721594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:48.721859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:48.723568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:48.770949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:48.841040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:48.842479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee9d5b-4a4a-4e4d-9d8b-f000a03bb722 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:48.862588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 679/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:52.873367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:52.873930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:52.873930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:52.875333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 668/1347] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:52.881330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a93b-d023-409a-a7f6-17d7e1517c3c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:52.881864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a93b-d023-409a-a7f6-17d7e1517c3c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:52.881864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a93b-d023-409a-a7f6-17d7e1517c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:52.954691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9759a93b-d023-409a-a7f6-17d7e1517c3c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:52.955965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9759a93b-d023-409a-a7f6-17d7e1517c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:53.059992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:53.062827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 680/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:41:53.340748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:41:53.342339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:53.451160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198461c3-45dd-41c2-a9f9-665f3a5cf2d8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:41:53.499438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:53.526853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:41:53.527509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:53.527795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:53.528056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:53.529641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:53.652808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:41:53.654248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5ebbf-9ac0-4ac8-8bf2-0fce8e88dcda service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:41:53.672668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 669/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:41:58.511514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713a04-a021-4c20-8e8a-fab812c6aaf8 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:58.513626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713a04-a021-4c20-8e8a-fab812c6aaf8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:58.514073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713a04-a021-4c20-8e8a-fab812c6aaf8 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:58.515914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 681/1350] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:41:58.523149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6694-49a1-4e10-a9e4-3c0821bd619f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:41:58.523599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6694-49a1-4e10-a9e4-3c0821bd619f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:41:58.523780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6694-49a1-4e10-a9e4-3c0821bd619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:41:58.652541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d23f6694-49a1-4e10-a9e4-3c0821bd619f None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:41:58.653668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d23f6694-49a1-4e10-a9e4-3c0821bd619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:41:58.687897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:41:58.793828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 670/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:41:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:02.790151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902178c-8928-4b58-b272-90944c406895 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:02.791124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902178c-8928-4b58-b272-90944c406895 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:02.791347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902178c-8928-4b58-b272-90944c406895 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:02.793060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 682/1352] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:02.797831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db636823-f0b9-4d8f-88c5-862f9f0ea1ca None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:02.798284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db636823-f0b9-4d8f-88c5-862f9f0ea1ca None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:02.798435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db636823-f0b9-4d8f-88c5-862f9f0ea1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:02.879507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-db636823-f0b9-4d8f-88c5-862f9f0ea1ca None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:02.881076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-db636823-f0b9-4d8f-88c5-862f9f0ea1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:02.954360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:02.996046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 671/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:42:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:04.473946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307339f-37e9-44d7-8ffa-38aa67e62113 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:04.474471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307339f-37e9-44d7-8ffa-38aa67e62113 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:04.474471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307339f-37e9-44d7-8ffa-38aa67e62113 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:04.475915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 683/1354] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:04.481282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a66947-78da-4209-9736-cda15d412477 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:04.481693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a66947-78da-4209-9736-cda15d412477 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:04.481856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a66947-78da-4209-9736-cda15d412477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:04.558887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-00a66947-78da-4209-9736-cda15d412477 None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:04.560149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-00a66947-78da-4209-9736-cda15d412477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:04.679362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 672/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:07.985826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05421-56ce-4852-ba60-6701e33cb62e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:07.986385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05421-56ce-4852-ba60-6701e33cb62e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:07.986509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05421-56ce-4852-ba60-6701e33cb62e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:07.987933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 684/1356] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:07.993504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17fbf0-e879-497c-bdc2-40a20f28d1f1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:07.994289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17fbf0-e879-497c-bdc2-40a20f28d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:07.994561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17fbf0-e879-497c-bdc2-40a20f28d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:08.035853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:08.079405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3e17fbf0-e879-497c-bdc2-40a20f28d1f1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:08.080618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3e17fbf0-e879-497c-bdc2-40a20f28d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:08.210919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 673/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:08.533734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:08.534390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:08.534790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:08.536257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 685/1358] 10.222.0.25 () {54 vars in 962 bytes} [Sat Sep 3 03:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:08.544032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1be1-4081-4bb1-b305-36c078dae2f7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:08.544851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1be1-4081-4bb1-b305-36c078dae2f7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:08.545243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1be1-4081-4bb1-b305-36c078dae2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:08.625017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e99d1be1-4081-4bb1-b305-36c078dae2f7 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:08.626218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e99d1be1-4081-4bb1-b305-36c078dae2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:08.676859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:08.747981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 674/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Sep 3 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:08.755743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:08.757503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:08.906405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d8b68-3d8e-47d9-9777-71d04562eab0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:08.986916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:09.031586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:09.032206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:09.032486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:09.032756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:09.034383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:09.132437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:09.134054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae628-d32a-458a-a32c-7ac2fa745e1e service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:09.155947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 686/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:42:11.270337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:11.270864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:11.270976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:11.272448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 675/1361] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:11.277531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0ac4-8ef2-4c4c-9984-8e27c90e0f88 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:11.277920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0ac4-8ef2-4c4c-9984-8e27c90e0f88 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:11.278096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0ac4-8ef2-4c4c-9984-8e27c90e0f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:11.353047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-415e0ac4-8ef2-4c4c-9984-8e27c90e0f88 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:11.354236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-415e0ac4-8ef2-4c4c-9984-8e27c90e0f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:11.392696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:11.454863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 687/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:11.754980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:11.756494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:11.910497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d139-92bc-4d6b-a2c1-088305591983 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:11.920335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:12.030244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:12.031028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:12.031417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:12.031789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:12.033598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:12.147423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:12.149101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece07bd-402f-4c2a-aed9-e80948763d7d service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:12.168846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 676/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:14.417531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2c6c-0388-4131-9f43-84d7a46ecdaa None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:14.418081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2c6c-0388-4131-9f43-84d7a46ecdaa None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:14.418081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2c6c-0388-4131-9f43-84d7a46ecdaa None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:14.419392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 688/1364] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:14.424917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8545d-3eec-44b5-a41c-1b7f65034821 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:14.425378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8545d-3eec-44b5-a41c-1b7f65034821 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:14.425461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8545d-3eec-44b5-a41c-1b7f65034821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:14.442152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:14.501760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-37e8545d-3eec-44b5-a41c-1b7f65034821 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:14.503069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-37e8545d-3eec-44b5-a41c-1b7f65034821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:14.604382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 677/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:14.893096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:14.893481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:14.893761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:14.895251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 689/1366] 10.222.0.25 () {54 vars in 962 bytes} [Sat Sep 3 03:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:14.900978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66a6a-ea3f-4592-9186-2aa1456f0a94 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:14.901264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66a6a-ea3f-4592-9186-2aa1456f0a94 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:14.901536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66a6a-ea3f-4592-9186-2aa1456f0a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:14.980710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0bc66a6a-ea3f-4592-9186-2aa1456f0a94 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:14.982016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0bc66a6a-ea3f-4592-9186-2aa1456f0a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:15.018746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:15.105488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 678/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Sep 3 03:42:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:15.113391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:15.114926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:15.275712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b1db3-34fd-4423-a337-e19ff9128d66 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:15.297287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:15.397855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:15.398465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:15.398743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:15.399052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:15.400629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:15.496358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:15.497952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff85e1-abec-4a7a-a6af-893af3228725 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:15.517363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 690/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:16.067299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55882f1-06c0-4e9e-aeb5-1d80ffefd5dc None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:16.067837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55882f1-06c0-4e9e-aeb5-1d80ffefd5dc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:16.067837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55882f1-06c0-4e9e-aeb5-1d80ffefd5dc None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:16.070849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 679/1369] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:16.074355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c02713-71d1-4d38-8a56-9e2f8cc34c40 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:16.074641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c02713-71d1-4d38-8a56-9e2f8cc34c40 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:16.074953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c02713-71d1-4d38-8a56-9e2f8cc34c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:16.107631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:16.144913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d7c02713-71d1-4d38-8a56-9e2f8cc34c40 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:16.146310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d7c02713-71d1-4d38-8a56-9e2f8cc34c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:16.247756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 691/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:16.828687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:16.829018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:16.829288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:16.830773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 680/1371] 10.222.0.25 () {54 vars in 964 bytes} [Sat Sep 3 03:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:16.835739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0512-8cbd-4bf3-bf65-ea682b6a5875 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:16.836219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0512-8cbd-4bf3-bf65-ea682b6a5875 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:16.836554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac0512-8cbd-4bf3-bf65-ea682b6a5875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:16.914925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b7ac0512-8cbd-4bf3-bf65-ea682b6a5875 None None] MFA Rules not processed for user `09249c2527c148cd99ebf899022f76fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:16.916165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b7ac0512-8cbd-4bf3-bf65-ea682b6a5875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:16.965154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:16.999998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 692/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Sat Sep 3 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:17.007708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:17.009252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:17.121500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a87fa-4caa-4d3f-b979-d9797bc99809 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:17.164173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:17.210834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:17.211866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:17.212326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:17.212995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:17.214835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:17.330054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:17.331577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d299c3-8c6c-4139-811f-0604458a4659 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:17.351520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 681/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:17.459547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:17.459901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:17.460208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:17.463083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 693/1374] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:17.466719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179a72-5693-42e1-a998-8ae7cce3c05b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:17.467046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179a72-5693-42e1-a998-8ae7cce3c05b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:17.467331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179a72-5693-42e1-a998-8ae7cce3c05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:17.543398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-89179a72-5693-42e1-a998-8ae7cce3c05b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:17.544717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-89179a72-5693-42e1-a998-8ae7cce3c05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:17.567546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:17.646477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 682/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:17.932091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:17.933605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:18.086916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245350d4-facb-42ee-8dd6-53dd7e097fa9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:18.139176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:18.200806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:18.201426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:18.201708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:18.201973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:18.203651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:18.296651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:18.298293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0df213-27da-40cb-a18b-d5fffb8912d1 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:18.316190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 694/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:20.252959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:20.253498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:20.253498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:20.254937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 683/1377] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:20.264792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931677b-99db-42b1-9713-2eed475f7b43 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:20.264792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931677b-99db-42b1-9713-2eed475f7b43 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:20.264792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931677b-99db-42b1-9713-2eed475f7b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:20.333425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0931677b-99db-42b1-9713-2eed475f7b43 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:20.334576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0931677b-99db-42b1-9713-2eed475f7b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:20.341177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:20.446840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 695/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:20.747451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:20.748989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:20.936722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaff0b6-08f6-415f-ab52-a4d064645d5a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:20.941651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:21.059449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:21.059970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:21.060250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:21.060517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:21.062096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:21.154468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:21.156101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd78031-34b7-45dc-a16b-618340daaea8 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:21.176818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 684/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:23.102087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:23.102604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:23.102902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:23.104468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 696/1380] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:23.109838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc55f8-38b5-4298-81f3-d89849f434a7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:23.110278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc55f8-38b5-4298-81f3-d89849f434a7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:23.110581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc55f8-38b5-4298-81f3-d89849f434a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:23.123028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:23.197013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f1bc55f8-38b5-4298-81f3-d89849f434a7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:23.198348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f1bc55f8-38b5-4298-81f3-d89849f434a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:23.304388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 685/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:23.591025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:23.592629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:23.750094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:23.761161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba087e-823b-49e4-9b47-4b7e30b27310 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:23.883817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:23.884427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:23.884774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:23.885131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:23.886781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:23.984086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:23.985781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac60b-3ff8-4a5f-a72b-d9ade641c92f service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:23.997923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:24.007404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 697/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:25.973720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:25.974325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:25.974443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:25.976249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 686/1383] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:25.982850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b3e7-2948-4a93-b79b-33cb0405ccf0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:25.983297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b3e7-2948-4a93-b79b-33cb0405ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:25.983554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b3e7-2948-4a93-b79b-33cb0405ccf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:26.065197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2940b3e7-2948-4a93-b79b-33cb0405ccf0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:26.066631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2940b3e7-2948-4a93-b79b-33cb0405ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:26.180955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 698/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:26.472777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:26.474655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:26.559485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:26.623471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e37de-eaa4-4cdc-996a-30ca13b28856 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:26.758338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:26.758949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:26.759338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:26.759631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:26.761629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:26.814820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:26.860403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:26.862665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dbcbc-99db-4836-86b0-c555aa7972c6 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:26.886405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 687/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:28.787265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:28.788015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:28.788015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:28.789540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 699/1386] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:28.795282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151970e2-aaac-486e-90f8-99c59350dd56 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:28.795757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151970e2-aaac-486e-90f8-99c59350dd56 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:28.795876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151970e2-aaac-486e-90f8-99c59350dd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:28.866761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-151970e2-aaac-486e-90f8-99c59350dd56 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:28.868345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-151970e2-aaac-486e-90f8-99c59350dd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:28.968634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 688/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:29.259653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:29.261242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:29.349510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:29.427708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173854aa-0a96-42ae-bad8-b5cd2d89878c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:29.544869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:29.545414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:29.545684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:29.545956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:29.547630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:29.586505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:29.639432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:29.641047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f81aa6-c52d-4ea4-b650-a23e0c8edf09 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:29.659229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 700/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:31.603912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:31.604491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:31.604491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:31.605733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 689/1389] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:31.611225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ba5a-f513-4b59-a13a-853b59b1f8a9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:31.611667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ba5a-f513-4b59-a13a-853b59b1f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:31.611779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ba5a-f513-4b59-a13a-853b59b1f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:31.807387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1d05ba5a-f513-4b59-a13a-853b59b1f8a9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:31.808618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1d05ba5a-f513-4b59-a13a-853b59b1f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:31.907273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 701/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:32.189966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:32.191569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:32.200436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dd35c-1a23-4a6e-b4b8-056c34f7a9e3 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:32.200726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dd35c-1a23-4a6e-b4b8-056c34f7a9e3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:32.201000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dd35c-1a23-4a6e-b4b8-056c34f7a9e3 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:32.202351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 702/1391] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:42:32.207115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4152-6b9e-461f-9e4d-7239a8fdb9b0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:32.207391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4152-6b9e-461f-9e4d-7239a8fdb9b0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:32.207665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4152-6b9e-461f-9e4d-7239a8fdb9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:32.224312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:32.225142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:32.278619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ba4e4152-6b9e-461f-9e4d-7239a8fdb9b0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:32.279887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ba4e4152-6b9e-461f-9e4d-7239a8fdb9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:32.345146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2b72c-5937-47d1-8e6f-0815ec844961 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:32.378148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 703/1392] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:42:32.449742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:32.462035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:32.462797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:32.463226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:32.463531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:32.465143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:32.556687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:32.558310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923b2d9-de10-448c-a281-66d262b8da75 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:32.576784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 690/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:32.673491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:32.674148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:32.674247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:32.675845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 704/1394] 10.222.0.25 () {54 vars in 961 bytes} [Sat Sep 3 03:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:32.681797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707eb73-54e5-44ab-b788-2528ce2083ed None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:32.682408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707eb73-54e5-44ab-b788-2528ce2083ed None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:32.682408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707eb73-54e5-44ab-b788-2528ce2083ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:32.769024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3707eb73-54e5-44ab-b788-2528ce2083ed None None] MFA Rules not processed for user `716e5d4773d643edbce5191347661f60`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:32.770182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3707eb73-54e5-44ab-b788-2528ce2083ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:32.824195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:32.850767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 691/1395] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Sep 3 03:42:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:32.859724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:32.861244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:32.960667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:32.968171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aca09c-9b47-42ce-b518-2f9b47acaee6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:33.042692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:33.043252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:33.043525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:33.043791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:33.045373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:33.171057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:33.172493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9d29c-4b2f-4b7a-83a3-af041f70388c service glance] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:33.192254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 705/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:33.941194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd7fb-3df6-4b4b-9026-2a8d67346d35 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:33.941685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd7fb-3df6-4b4b-9026-2a8d67346d35 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:33.941875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd7fb-3df6-4b4b-9026-2a8d67346d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.010351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1c4bd7fb-3df6-4b4b-9026-2a8d67346d35 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:34.011580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1c4bd7fb-3df6-4b4b-9026-2a8d67346d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:34.096211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 692/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:42:34.136957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:34.137490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:34.137490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.138888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 706/1398] 10.222.0.25 () {54 vars in 943 bytes} [Sat Sep 3 03:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:34.145062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40a74a-2e63-46aa-98bf-718e2f5cb5cc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:34.145350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40a74a-2e63-46aa-98bf-718e2f5cb5cc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:34.145628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40a74a-2e63-46aa-98bf-718e2f5cb5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.221840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bd40a74a-2e63-46aa-98bf-718e2f5cb5cc None None] MFA Rules not processed for user `323c45a05bd241458486b8919c93dd2e`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:34.223290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bd40a74a-2e63-46aa-98bf-718e2f5cb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:34.239292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:34.333503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 693/1399] 10.222.0.25 () {58 vars in 1040 bytes} [Sat Sep 3 03:42:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:34.343057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:34.344650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:34.370335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:34.492068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2f48-a1b9-4eca-8b0d-50baed35361d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:34.520847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b147ee-e8e6-461b-9de7-4340adc4510e None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:34.521136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b147ee-e8e6-461b-9de7-4340adc4510e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:34.521409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b147ee-e8e6-461b-9de7-4340adc4510e None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.522763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 694/1400] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:42:34.527662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:34.527939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:34.528216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.592933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:34.594141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:34.597185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:34.597796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:34.598386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:34.598656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:34.600284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:34.607508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:34.670291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:34.685049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:34.686648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17ae80-5c03-4ef9-9a04-ee31e803a651 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:34.693425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 695/1401] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:42:34.695298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 707/1402] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:42:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:42:34.977948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:34.979187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:35.102095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e15ce-5216-4352-a89a-a570d03f9164 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:35.202335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:35.237754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:35.238464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:35.238740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:35.239068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:35.240683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:35.348651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:35.350258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7591aca-0b0a-4aac-8167-8224284cc853 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:35.369825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 696/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:37.303642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:37.304190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:37.304190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:37.305568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 708/1404] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:37.310738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914f9-0fb5-496a-ab3b-fb02702ec5f8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:37.311203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914f9-0fb5-496a-ab3b-fb02702ec5f8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:37.311302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914f9-0fb5-496a-ab3b-fb02702ec5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:37.388332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-10c914f9-0fb5-496a-ab3b-fb02702ec5f8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:37.389638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-10c914f9-0fb5-496a-ab3b-fb02702ec5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:37.444519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:37.494705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 697/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:37.819145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:37.820673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:37.979093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd67f7-551e-4725-a226-4910fb5119ff None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:37.982045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:38.098282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:38.099356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:38.099842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:38.100108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:38.101721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:38.202828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:38.204495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c469f-ec88-4f4a-82c7-21a140a03c0d service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:38.224015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 709/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:40.173271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:40.173878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:40.173878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:40.176363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 698/1407] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:40.180446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c5b-3f22-4e65-96a5-e0990eda6ff0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:40.182646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c5b-3f22-4e65-96a5-e0990eda6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:40.182951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9c5b-3f22-4e65-96a5-e0990eda6ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:40.193413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:40.269680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f04c9c5b-3f22-4e65-96a5-e0990eda6ff0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:40.270945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f04c9c5b-3f22-4e65-96a5-e0990eda6ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:40.381872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 710/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:40.678156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:40.679771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:40.847394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c138-034e-4760-b3e9-9c87c48fd13d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:40.893133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:40.979350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:40.979960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:40.980238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:40.980513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:40.982163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:41.078047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:41.079704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ade8e5-d1e9-4be0-a869-33a47d6e6504 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:41.101657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 699/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:42:50.380290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=62 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:51.099214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=82 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:56.278263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381223b1-25b5-47b9-8d7e-1f21335f1b9d None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:56.278773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381223b1-25b5-47b9-8d7e-1f21335f1b9d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:56.278773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381223b1-25b5-47b9-8d7e-1f21335f1b9d None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:56.280237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 711/1410] 10.222.0.25 () {54 vars in 918 bytes} [Sat Sep 3 03:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:56.285665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2a57-4583-4896-8720-2ad647055559 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:56.286108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2a57-4583-4896-8720-2ad647055559 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:56.286238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2a57-4583-4896-8720-2ad647055559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:56.366846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b28b2a57-4583-4896-8720-2ad647055559 None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:56.368079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b28b2a57-4583-4896-8720-2ad647055559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:56.478639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 700/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 3 03:42:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:56.494224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef9705-a67b-4294-924a-7f58b79548f7 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:56.494506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef9705-a67b-4294-924a-7f58b79548f7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:56.494779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef9705-a67b-4294-924a-7f58b79548f7 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:56.496183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 712/1412] 10.222.0.25 () {54 vars in 918 bytes} [Sat Sep 3 03:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:56.501995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cc9f0-a8ba-4074-812b-162afca3729f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:56.502492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cc9f0-a8ba-4074-812b-162afca3729f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:56.502777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cc9f0-a8ba-4074-812b-162afca3729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:56.580091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:56.584386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-524cc9f0-a8ba-4074-812b-162afca3729f None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:56.586741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-524cc9f0-a8ba-4074-812b-162afca3729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:56.712440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 701/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 3 03:42:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:59.061454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489694-28ea-40a5-917f-a167952dca9b None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:59.062052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489694-28ea-40a5-917f-a167952dca9b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:59.062052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489694-28ea-40a5-917f-a167952dca9b None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:59.064210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:42:59.072949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab936-8e46-4753-87f5-a24f22f8e03f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:59.073364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab936-8e46-4753-87f5-a24f22f8e03f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:59.073537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab936-8e46-4753-87f5-a24f22f8e03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:59.147794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bb8ab936-8e46-4753-87f5-a24f22f8e03f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:42:59.148994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bb8ab936-8e46-4753-87f5-a24f22f8e03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:59.210174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:59.254007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:42:59.541741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489694-28ea-40a5-917f-a167952dca9b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:42:59.543998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-57489694-28ea-40a5-917f-a167952dca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:59.704220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489694-28ea-40a5-917f-a167952dca9b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:42:59.783083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:42:59.833199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:42:59.833824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:42:59.834112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:42:59.834368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:42:59.836023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:42:59.940196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:42:59.941783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9644d-09e4-43e4-8bb7-a39b5ff013e4 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:42:59.961235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 714/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:00.838422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:00.839395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:00.839395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:00.841494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:00.847169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745bed6-1dcd-44f7-92f7-bb8865018582 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:00.847637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745bed6-1dcd-44f7-92f7-bb8865018582 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:00.847745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745bed6-1dcd-44f7-92f7-bb8865018582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:00.929950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e745bed6-1dcd-44f7-92f7-bb8865018582 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:00.931206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e745bed6-1dcd-44f7-92f7-bb8865018582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:00.960117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:01.039677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:01.188294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:01.189816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:01.354115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1001f8-0046-4f97-afc6-d1006a3b76cc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:01.395784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:01.470572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:01.471129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:01.471438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:01.471697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:01.473273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:01.568999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:01.570624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118efb6-486b-4749-9bba-744e21095341 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:01.595711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 704/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:02.484686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:02.485264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:02.485264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:02.486666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:02.492780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f87f4-372d-4509-8e7b-20f5ed0a72fa None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:02.493520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f87f4-372d-4509-8e7b-20f5ed0a72fa None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:02.493658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f87f4-372d-4509-8e7b-20f5ed0a72fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:02.553988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:02.576459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c19f87f4-372d-4509-8e7b-20f5ed0a72fa None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:02.577670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c19f87f4-372d-4509-8e7b-20f5ed0a72fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:02.677879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:02.826740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:02.829229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:02.991871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee259bb-d1db-43cb-b3ce-2dd6db4867ec None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:03.004030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:03.110608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:03.111157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:03.111435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:03.111706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:03.113349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:03.206573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:03.208210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8de75-97d2-4ee4-a629-7b7460d12da8 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:03.227640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 717/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:04.095193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:04.095754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:04.095754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:04.097082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:04.103139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc621e-3f93-40f2-a86b-2c3527902b55 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:04.103583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc621e-3f93-40f2-a86b-2c3527902b55 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:04.103891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc621e-3f93-40f2-a86b-2c3527902b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:04.132011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:04.190223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-55cc621e-3f93-40f2-a86b-2c3527902b55 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:04.191467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-55cc621e-3f93-40f2-a86b-2c3527902b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:04.293604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:04.447408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:04.448945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:04.567071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:04.604486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea08e9-e670-4186-bc69-aa129e8eccef None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:04.732923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:04.733526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:04.733805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:04.734073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:04.735715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:04.817972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:04.834636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:04.836279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb480f0-32f7-4c0e-a8d1-4cb604145b17 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:04.855887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 707/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:05.735933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:05.736521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:05.736521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:05.737906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:05.744226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef0ae-ffe5-48dd-ba3a-762ddb8c5ac3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:05.744939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef0ae-ffe5-48dd-ba3a-762ddb8c5ac3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:05.745095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef0ae-ffe5-48dd-ba3a-762ddb8c5ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:05.834263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c8eef0ae-ffe5-48dd-ba3a-762ddb8c5ac3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:05.835522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c8eef0ae-ffe5-48dd-ba3a-762ddb8c5ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:05.935095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:06.087674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:06.089929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:06.182765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:06.250560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c5075-47b0-4491-a2d8-9017c9056643 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:06.365627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:06.366147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:06.366423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:06.366692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:06.368312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:06.418879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:06.466373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:06.468009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f616e9-ce52-4ae2-be90-8c5dabf2dc0f service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:06.487842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 720/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:07.288037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:07.288566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:07.288566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:07.290027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:07.295235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d678-f39f-49ae-8c36-975ffee572e6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:07.295640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d678-f39f-49ae-8c36-975ffee572e6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:07.295853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d678-f39f-49ae-8c36-975ffee572e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:07.375336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5e92d678-f39f-49ae-8c36-975ffee572e6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:07.376561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5e92d678-f39f-49ae-8c36-975ffee572e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:07.486261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:07.636778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:07.638322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:07.689450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:07.789403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3417-0e05-4f96-8aa7-a90c9f949dc3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:07.905466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:07.906073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:07.906353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:07.906628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:07.908270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:07.915349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:08.006060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:08.007698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd462788-139b-4de3-a51a-65845398bd73 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:08.026744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 710/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:08.841426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:08.842112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:08.842112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:08.843367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:08.849902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c9ae-42ae-4025-b6db-76edd7250afc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:08.850695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c9ae-42ae-4025-b6db-76edd7250afc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:08.850918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c9ae-42ae-4025-b6db-76edd7250afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:08.939241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1f71c9ae-42ae-4025-b6db-76edd7250afc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:08.940532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1f71c9ae-42ae-4025-b6db-76edd7250afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:09.011124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:09.040145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:09.194731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:09.197081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:09.221647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:09.358030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50940f-1e9d-4edb-b2ea-74c3d9bcfcd5 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:09.454918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:09.488320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:09.488914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:09.489187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:09.489454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:09.491115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:09.588468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:09.590065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b0492-96aa-4267-91d1-7649db441212 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:09.609136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 723/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:10.419535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:10.420459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:10.420459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:10.422571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:10.428628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada641f1-82f2-4ecf-80ea-7e2d642b9d89 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:10.429070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada641f1-82f2-4ecf-80ea-7e2d642b9d89 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:10.429183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada641f1-82f2-4ecf-80ea-7e2d642b9d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:10.518670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ada641f1-82f2-4ecf-80ea-7e2d642b9d89 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:10.519869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ada641f1-82f2-4ecf-80ea-7e2d642b9d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:10.562145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:10.622667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:10.787680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:10.789288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:10.937696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b97ba1-c130-42cf-aef1-f7113f50c819 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:10.992630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:11.056938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:11.057463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:11.057737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:11.057997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:11.059638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:11.152449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:11.154046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d90009-b824-4754-aefb-6ac89f2e8b66 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:11.174762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 713/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:11.980454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536807a-480a-4027-9623-4495f9bea402 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:11.981330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536807a-480a-4027-9623-4495f9bea402 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:11.981330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536807a-480a-4027-9623-4495f9bea402 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:11.983618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:43:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:11.988733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7bab-c019-48e4-a46b-a1334a857bb0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:11.989186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7bab-c019-48e4-a46b-a1334a857bb0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:11.989295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7bab-c019-48e4-a46b-a1334a857bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:12.067219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2fed7bab-c019-48e4-a46b-a1334a857bb0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:12.068414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2fed7bab-c019-48e4-a46b-a1334a857bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:12.070354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:12.174012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:43:12.325058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536807a-480a-4027-9623-4495f9bea402 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:12.326594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1536807a-480a-4027-9623-4495f9bea402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:12.497252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536807a-480a-4027-9623-4495f9bea402 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:12.523572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:12.617037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:12.617639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:12.617942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:12.618204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:12.619798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:12.734254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:12.736553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b927fa-3cbc-43fc-ab0d-3a7f203aa4fd service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:12.759887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 726/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:14.567352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:14.568220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:14.568220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:14.570435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:14.576062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5246-2910-49b6-b0e2-f39749453584 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:14.576495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5246-2910-49b6-b0e2-f39749453584 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:14.576586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5246-2910-49b6-b0e2-f39749453584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:14.624362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:14.651107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b23e5246-2910-49b6-b0e2-f39749453584 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:14.652317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b23e5246-2910-49b6-b0e2-f39749453584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:14.754311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:15.038785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:15.041285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:15.161530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eceeb3-502b-4e2d-b63d-b88a111bfeca None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:15.195265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:15.240083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:15.240725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:15.240997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:15.241262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:15.242886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:15.353354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:15.354728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8d882-9788-4ab7-be2b-e445863b2313 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:15.376812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 716/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:17.410325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:17.410946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:17.410946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:17.412301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:17.418833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ed4e-cc36-4b6e-afe0-42de0d3fbcdb None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:17.419586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ed4e-cc36-4b6e-afe0-42de0d3fbcdb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:17.419776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ed4e-cc36-4b6e-afe0-42de0d3fbcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:17.496250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0ec0ed4e-cc36-4b6e-afe0-42de0d3fbcdb None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:17.497465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0ec0ed4e-cc36-4b6e-afe0-42de0d3fbcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:17.510895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:17.598968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:17.895778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:17.898226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:18.014720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10414ae0-8e3c-40e6-8b12-2fe23a2b3719 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:18.027113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:18.089216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:18.089740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:18.090018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:18.090283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:18.091910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:18.211215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:18.212658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1465d2-e9cb-4b7a-b43b-79b7f0275a85 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:18.232303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 729/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:20.444119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:20.444697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:20.444697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:20.446039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:20.452556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2752-d862-4f48-be6f-ebe7e9327ca0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:20.453350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2752-d862-4f48-be6f-ebe7e9327ca0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:20.453490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2752-d862-4f48-be6f-ebe7e9327ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:20.509129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:20.527626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3aaa2752-d862-4f48-be6f-ebe7e9327ca0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:20.528834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3aaa2752-d862-4f48-be6f-ebe7e9327ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:20.641558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:20.925930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:20.928622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:21.059350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358745b-14ea-414c-9c10-18e41439d49f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:21.107614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:21.149147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:21.149757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:21.150037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:21.150301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:21.152023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:21.265797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:21.267290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ed5e-0f08-4c32-8045-d7b5474b8c1d service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:21.285959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 719/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:23.354744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:23.355357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:23.355357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:23.356675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:23.363000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01404fb-7172-45f3-8a06-a36808c83038 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:23.363750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01404fb-7172-45f3-8a06-a36808c83038 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:23.363871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01404fb-7172-45f3-8a06-a36808c83038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:23.446461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f01404fb-7172-45f3-8a06-a36808c83038 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:23.447671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f01404fb-7172-45f3-8a06-a36808c83038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:23.468959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:23.567505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:23.855104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:23.856680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:23.976240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5ee39-66f7-4da2-8a4d-06e72c6deecd None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:23.996304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:24.056871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:24.057346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:24.057550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:24.057824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:24.059495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:24.182199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:24.183595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc784d5-9ddb-468b-9819-87efd593b911 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:24.204730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 732/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:26.155178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:26.155693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:26.155693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:26.157751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:26.165817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3781f-bcc7-4897-ac3c-cfeec84b70a4 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:26.166301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3781f-bcc7-4897-ac3c-cfeec84b70a4 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:26.166301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3781f-bcc7-4897-ac3c-cfeec84b70a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:26.230353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:26.233757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-70f3781f-bcc7-4897-ac3c-cfeec84b70a4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:26.235637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-70f3781f-bcc7-4897-ac3c-cfeec84b70a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:26.343045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:43:26.619783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:26.621355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:26.729167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf336c-e60c-4288-b8ce-d9f5b1b34346 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:26.779515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:26.811681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:26.812288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:26.812560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:26.812828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:26.814413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:26.932126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:26.933517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d28b38-9162-49ad-8aa8-f15aad6ee28d service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:26.952982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 722/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:28.931669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:28.932574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:28.932574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:28.934683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:28.940961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95693e-71c6-47f1-8901-66de7bb4e1e7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:28.941435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95693e-71c6-47f1-8901-66de7bb4e1e7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:28.941534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95693e-71c6-47f1-8901-66de7bb4e1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:29.009078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-cf95693e-71c6-47f1-8901-66de7bb4e1e7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:29.010274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cf95693e-71c6-47f1-8901-66de7bb4e1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:29.039858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:29.114826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:29.400880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:29.403492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:29.513191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6d299-55a9-4b9e-a8be-5e0d4e2f5818 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:29.541381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:29.606284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:29.606936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:29.607226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:29.607494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:29.609162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:29.738902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:29.740353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35aeaa0-a61c-4af6-ac40-83aaad7dbd0d service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:29.760776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 735/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:31.686868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:31.687451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:31.687451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:31.688787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:31.695018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d69d-93d3-491c-ae25-3c95b5da25a3 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:31.695471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d69d-93d3-491c-ae25-3c95b5da25a3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:31.695591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d69d-93d3-491c-ae25-3c95b5da25a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:31.776782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e546d69d-93d3-491c-ae25-3c95b5da25a3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:31.777996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e546d69d-93d3-491c-ae25-3c95b5da25a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:31.786816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:31.879355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:32.162897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:32.165457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:32.281372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6bbc5-c2a4-4ae5-92ac-e1aadbaa6348 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:32.337786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:32.358053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:32.358668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:32.358970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:32.359243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:32.360831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:32.482214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:32.484486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984d0ec-de7e-45c5-a2a4-3e6b9ac0f856 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:32.511176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 725/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:34.455435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:34.456013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:34.456013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:34.457421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:34.463807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92b3c5-1c19-4422-b74f-d33a29fcd1a1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:34.464274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92b3c5-1c19-4422-b74f-d33a29fcd1a1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:34.464383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92b3c5-1c19-4422-b74f-d33a29fcd1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:34.533245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8a92b3c5-1c19-4422-b74f-d33a29fcd1a1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:34.534366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8a92b3c5-1c19-4422-b74f-d33a29fcd1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:34.573939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:34.642827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:34.926002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:34.927618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:35.052047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e004c7-1713-4ac6-85cc-9eda62913437 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:35.090679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:35.130853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:35.131488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:35.131766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:35.132039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:35.133647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:35.267757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:35.269304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c51a83-2ef8-48e6-a884-a71f3cdaaca5 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:35.289914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 738/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:37.190382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:37.191236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:37.191236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:37.192539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:37.198302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dbe0-0ab6-43c2-9a5a-ffdc5ef67cf2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:37.198773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dbe0-0ab6-43c2-9a5a-ffdc5ef67cf2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:37.198893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dbe0-0ab6-43c2-9a5a-ffdc5ef67cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:37.265233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-78e4dbe0-0ab6-43c2-9a5a-ffdc5ef67cf2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:37.266450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-78e4dbe0-0ab6-43c2-9a5a-ffdc5ef67cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:37.284482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:37.372674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:43:37.658170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:37.659939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:37.780145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c81d2-718b-483d-bbb0-194fca69facb None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:37.853191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:37.861149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:37.861774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:37.862099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:37.862302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:37.864425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:37.976531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:37.977958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d345048-ce9f-4f6a-b4ef-b3615c26d50d service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:37.996191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 728/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:39.904269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:39.904884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:39.904884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:39.906219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:43:39.912783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d24de-5b1c-4135-bba8-fad51c950980 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:39.913520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d24de-5b1c-4135-bba8-fad51c950980 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:39.913703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d24de-5b1c-4135-bba8-fad51c950980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:39.994529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-719d24de-5b1c-4135-bba8-fad51c950980 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:43:39.995768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-719d24de-5b1c-4135-bba8-fad51c950980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:40.047385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:40.098069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:43:40.390929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:43:40.393327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:40.508480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef1636-16dc-4182-af23-dd496c1f70d1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:43:40.558859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:40.595266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:43:40.595918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:43:40.596199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:43:40.596466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:43:40.598589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:43:40.718412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:43:40.719927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93ce34-6546-4194-82ee-ef5c01cfb98f service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:43:40.739795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 741/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:43:50.096329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=22 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:43:50.737007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=66 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:00.095698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170c22-d6ae-4aa3-8504-4cfcbe23d2d9 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:00.096555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170c22-d6ae-4aa3-8504-4cfcbe23d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:00.096555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170c22-d6ae-4aa3-8504-4cfcbe23d2d9 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:00.098963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 730/1471] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:00.105710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bbd2a-c414-4816-8f42-17a978f00ebe None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:00.106197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bbd2a-c414-4816-8f42-17a978f00ebe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:00.106309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bbd2a-c414-4816-8f42-17a978f00ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:00.193229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-924bbd2a-c414-4816-8f42-17a978f00ebe None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:00.194444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-924bbd2a-c414-4816-8f42-17a978f00ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:00.301393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 742/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:00.600765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:00.601073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:00.601349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:00.603108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 731/1473] 10.222.0.25 () {54 vars in 961 bytes} [Sat Sep 3 03:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:00.608223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d3a4-0242-48c6-86ee-8bb9628aa4ad None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:00.608482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d3a4-0242-48c6-86ee-8bb9628aa4ad None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:00.608754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d3a4-0242-48c6-86ee-8bb9628aa4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:00.687745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8924d3a4-0242-48c6-86ee-8bb9628aa4ad None None] MFA Rules not processed for user `716e5d4773d643edbce5191347661f60`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:00.688960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8924d3a4-0242-48c6-86ee-8bb9628aa4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:00.711861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:00.768850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 743/1474] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Sep 3 03:44:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:00.777063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:00.779472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:00.898151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331c45a-50a9-4507-ab52-b1a9962f1cff None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:00.973700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:00.974306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:00.974587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:00.974852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:00.976469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:01.031977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:01.090584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:01.092096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb4bdc-e225-4aa2-ae84-c3b165751e17 service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:01.113039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 732/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:44:03.079051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:03.079938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:03.079938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:03.082120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 744/1476] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:03.088379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8fc5-63db-4faa-b626-75d404a5d5b4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:03.089075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8fc5-63db-4faa-b626-75d404a5d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:03.089230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8fc5-63db-4faa-b626-75d404a5d5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:03.164025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-774e8fc5-63db-4faa-b626-75d404a5d5b4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:03.165795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-774e8fc5-63db-4faa-b626-75d404a5d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:03.276286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 733/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:03.540496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:03.542968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:03.681977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515bf7-c08c-4d72-94d1-b7ec5b469538 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:03.730655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:03.779757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:03.780675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:03.781038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:03.781472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:03.782457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:03.785277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:44:03.785277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:44:03.785277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:44:03.787101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8870efd-b1dd-431c-9fc4-098ce5f7dd46 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:03.810862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 745/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Sep 3 03:44:03 2022] GET /identity/v3/projects => generated 2553 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:44:05.702643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:05.703658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:05.703658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:05.705635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 734/1479] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:05.713645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ef29-af39-4d66-8ecf-f92afa113864 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:05.714086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ef29-af39-4d66-8ecf-f92afa113864 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:05.714190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ef29-af39-4d66-8ecf-f92afa113864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:05.784597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4ac5ef29-af39-4d66-8ecf-f92afa113864 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:05.786006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4ac5ef29-af39-4d66-8ecf-f92afa113864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:05.893220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 746/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:44:06.183489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:06.185477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:06.197741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:06.337734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ad841-3b30-4c5e-91ed-584d85662859 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:06.420449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:06.455174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:06.455776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:06.456045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:06.456306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:06.457970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:06.547155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:06.548775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2f15-f582-4fd8-a63e-74c3f5b08bd8 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:06.566936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 735/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:44:08.623809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:08.624344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:08.624344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:08.625593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 747/1482] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:08.631636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e58d3-7429-4a8f-a590-69c5b89778ff None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:08.632297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e58d3-7429-4a8f-a590-69c5b89778ff None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:08.632405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e58d3-7429-4a8f-a590-69c5b89778ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:08.715207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a89e58d3-7429-4a8f-a590-69c5b89778ff None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:08.717038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a89e58d3-7429-4a8f-a590-69c5b89778ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:08.781590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:08.834672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 736/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:09.119253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:09.121721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:09.149626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:09.234091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e4ee6-75d3-4f00-b048-1c7057580416 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:09.310654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:09.311318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:09.311603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:09.311874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:09.313496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:09.392530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:09.429155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:09.430616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c932-2d23-4144-9b2a-0c9ed4edb0ec service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:09.451661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 748/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:44:11.420496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:11.421074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:11.421074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:11.422362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 737/1485] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:11.428090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8dff9-5358-42d6-845e-9bf4f283e73d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:11.428554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8dff9-5358-42d6-845e-9bf4f283e73d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:11.428657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8dff9-5358-42d6-845e-9bf4f283e73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:11.495990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-76e8dff9-5358-42d6-845e-9bf4f283e73d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:11.497215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-76e8dff9-5358-42d6-845e-9bf4f283e73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:11.595024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 749/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:44:11.889268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:11.890849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:12.030974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fdbcf-91cc-4114-bba6-91545641219a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:12.117102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:12.120555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:12.121475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:12.121846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:12.122213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:12.125758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:12.240356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:12.241834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57208c5a-4994-499f-974c-4ec4bd78864c service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:12.261003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 738/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:44:14.355082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:14.355733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:14.355733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:14.357782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 750/1488] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:14.363161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efac971-5ccd-4a07-a30b-3759133d2c29 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:14.363622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efac971-5ccd-4a07-a30b-3759133d2c29 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:14.363737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efac971-5ccd-4a07-a30b-3759133d2c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:14.430919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8efac971-5ccd-4a07-a30b-3759133d2c29 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:14.432066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8efac971-5ccd-4a07-a30b-3759133d2c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:14.485302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:14.536658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 739/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:14.827755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:14.829329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:14.869988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:14.940187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789bc1f-e564-4d23-a18b-61986552df7b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:15.016159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:15.016776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:15.017027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:15.017324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:15.018954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:15.101159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:15.134681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:15.136144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b78-354f-479a-a3c2-84645081f4af service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:15.154600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 751/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:44:17.261544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:17.261992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:17.262148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:17.263469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 740/1491] 10.222.0.25 () {54 vars in 927 bytes} [Sat Sep 3 03:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:44:17.268810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f36a-5637-4670-828e-13a4f5709ec2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:17.269290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f36a-5637-4670-828e-13a4f5709ec2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:17.269391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f36a-5637-4670-828e-13a4f5709ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:17.337397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b686f36a-5637-4670-828e-13a4f5709ec2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:44:17.338612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b686f36a-5637-4670-828e-13a4f5709ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:17.438459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 752/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Sep 3 03:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:44:17.727792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:44:17.729987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:17.838510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f919fac-e129-4d40-af3b-10bf8c28fa5c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:44:17.921657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:44:17.922287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:44:17.922526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:44:17.922797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:44:17.924423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:44:17.933573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:18.053797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:44:18.055254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12ef97-c049-4dcf-916e-e3ff1aad3052 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:44:18.074624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 741/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:44:27.436400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=82 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:44:28.071938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=50 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:37.019306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:37.019934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:37.019934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:37.105246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:45:37.106466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:37.207097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 753/1494] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:45:37.216118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:37.216653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:37.216653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:37.301912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:45:37.303115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:37.401100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 742/1495] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:45:37.405873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:37.407121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:37.418843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7f199-51bf-44e6-9f84-a7f406289a38 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:37.489650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:37.526345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:37.526993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:37.527273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:37.527541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:37.528183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:45:37.528486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:37.529505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:37.529505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:45:37.529505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:45:37.530406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:37.541614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 754/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:45:37.550252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:37.559758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4716d-6485-45ce-a6bc-898f62f19f7d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:37.622664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:37.658755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:37.659388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:37.659664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:37.659929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:37.660564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:37.662443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:37.662443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:45:37.662443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:45:37.663687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:37.722918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 743/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:45:37 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:45:37.727256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:37.728805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:37.764331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2066b-adf3-48b5-83ce-c2e1957ca024 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:37.864115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:37.864666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:37.864940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:37.865205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:37.865830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:37.866855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:37.866855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:45:37.866855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:45:37.868077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:37.920120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 755/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:45:37 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:45:37.924485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:37.926069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:37.992486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be8443-b6db-48e0-8224-e28d52c23d2c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:38.077569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:38.089810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:38.090417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.090694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.090998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.091652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:38.092688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:38.092688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:45:38.092688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:45:38.093585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:38.112065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 744/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:45:38.116535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:38.118026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:38.129850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356380c3-d11a-4c48-9175-ab6019fbee49 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:38.156685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:38.230538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:38.231180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.231488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.231758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.232412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:38.233423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:38.233423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:45:38.233423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:45:38.234317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b0f2e-a459-49bf-9a7f-88e61fbbf290 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:38.247465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 756/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:45:38.251895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:38.253456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:38.291072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81323a19-bd79-45f3-bf67-29174dd537ba admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:38.410711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:38.411589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.411589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.411889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] PATH_INFO: `/v3/projects/eee2ca2650b54c19a091fd3e938f170f/users/199f86330c63495e8dc0220ffc01ff50/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.436912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee2ca2650b54c19a091fd3e938f170f, user_id=199f86330c63495e8dc0220ffc01ff50, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:38.438524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:38.438524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:45:38.438524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:45:38.441521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c89eb-66a8-4c59-833c-6436695b2431 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:38.470998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 745/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:45:38 2022] PUT /identity/v3/projects/eee2ca2650b54c19a091fd3e938f170f/users/199f86330c63495e8dc0220ffc01ff50/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:45:38.484027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566406-09d3-418d-b37a-29049bcf5caa None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.484332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566406-09d3-418d-b37a-29049bcf5caa None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.484593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566406-09d3-418d-b37a-29049bcf5caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.544373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d7566406-09d3-418d-b37a-29049bcf5caa None None] MFA Rules not processed for user `199f86330c63495e8dc0220ffc01ff50`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:45:38.545648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d7566406-09d3-418d-b37a-29049bcf5caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:38.632127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 757/1502] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:45:38.649782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.650139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.650513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.652604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 746/1503] 10.222.0.25 () {54 vars in 964 bytes} [Sat Sep 3 03:45:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:45:38.659140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f1e91-3586-4b1c-a4ea-df00be77205b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:38.659429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f1e91-3586-4b1c-a4ea-df00be77205b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:38.659700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f1e91-3586-4b1c-a4ea-df00be77205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:38.696084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:38.754229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-083f1e91-3586-4b1c-a4ea-df00be77205b None None] MFA Rules not processed for user `09249c2527c148cd99ebf899022f76fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:45:38.755513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-083f1e91-3586-4b1c-a4ea-df00be77205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:38.847481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 758/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Sat Sep 3 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:45:38.855810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:38.857337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:38.909583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:38.967630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70023b4f-a470-4434-b902-6da6af84d8a2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:39.052875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:39.053486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:39.053765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:39.054034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:39.055672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:39.165096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:39.166539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:39.170067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:39.185885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 747/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:45:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:45:39.231253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a4e3-9ad9-4002-942f-acbc8380ac9e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:39.231697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a4e3-9ad9-4002-942f-acbc8380ac9e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:39.231806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a4e3-9ad9-4002-942f-acbc8380ac9e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:39.233143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 759/1506] 10.222.0.25 () {52 vars in 830 bytes} [Sat Sep 3 03:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:45:39.237140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:39.238406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:39.339540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee34f-933a-4d1e-a530-57ba2b11bc5a service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:39.561644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:39.562175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:39.562458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:39.562730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] PATH_INFO: `/v3/users/199f86330c63495e8dc0220ffc01ff50` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:39.577393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199f86330c63495e8dc0220ffc01ff50)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:39.578427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:39.578427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:45:39.578427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:45:39.579704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:39.590124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:39.655500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199f86330c63495e8dc0220ffc01ff50'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:39.655945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199f86330c63495e8dc0220ffc01ff50'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:39.680608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199f86330c63495e8dc0220ffc01ff50'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:39.694930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199f86330c63495e8dc0220ffc01ff50'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:39.695493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199f86330c63495e8dc0220ffc01ff50'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:39.712377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 748/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:45:39 2022] DELETE /identity/v3/users/199f86330c63495e8dc0220ffc01ff50 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:45:39.716878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847439e-b598-4132-bb02-8c42545be9db None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:39.717162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847439e-b598-4132-bb02-8c42545be9db None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:39.717434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847439e-b598-4132-bb02-8c42545be9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:39.779033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5847439e-b598-4132-bb02-8c42545be9db None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:45:39.780243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5847439e-b598-4132-bb02-8c42545be9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:39.861385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:39.875793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 760/1508] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:45:39.886571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:39.887857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:39.999542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef089e7-1080-41f1-b8f1-0fe8928c5366 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:40.058692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:40.079578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:40.080219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:40.080471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:40.080818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:40.082518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:40.201503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:40.202954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbebde9-7d21-4b9c-a91e-c6ddb57d9a3b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:40.221799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 749/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:45:40.606535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847439e-b598-4132-bb02-8c42545be9db None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:45:40.607909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5847439e-b598-4132-bb02-8c42545be9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:45:40.619262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847439e-b598-4132-bb02-8c42545be9db None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:45:40.717223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:45:40.717871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:45:40.718151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:45:40.718424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] PATH_INFO: `/v3/projects/eee2ca2650b54c19a091fd3e938f170f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:45:40.728869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee2ca2650b54c19a091fd3e938f170f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:45:40.730163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:45:40.730163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:45:40.730163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:45:40.731423 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:45:40.793034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee2ca2650b54c19a091fd3e938f170f', 'request_id': 'req-1b1eceab-1f2c-403d-8910-485208bd17df'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:40.802708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee2ca2650b54c19a091fd3e938f170f', 'request_id': 'req-1b1eceab-1f2c-403d-8910-485208bd17df'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:40.808873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee2ca2650b54c19a091fd3e938f170f', 'request_id': 'req-1b1eceab-1f2c-403d-8910-485208bd17df'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:40.839917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] The token cache is being invalidate because project eee2ca2650b54c19a091fd3e938f170f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:45:40.840205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:40.840721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b1eceab-1f2c-403d-8910-485208bd17df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:45:40.842080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 761/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:45:40 2022] DELETE /identity/v3/projects/eee2ca2650b54c19a091fd3e938f170f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:45:50.219445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=62 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:45:50.837359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:47:04.057801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:04.058433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:04.058433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:04.059726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:04.064856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbdcde-cfa9-424b-aacb-b2f7c4ca792a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:04.065322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbdcde-cfa9-424b-aacb-b2f7c4ca792a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:04.065424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbdcde-cfa9-424b-aacb-b2f7c4ca792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:04.148825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7fbbdcde-cfa9-424b-aacb-b2f7c4ca792a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:04.150013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7fbbdcde-cfa9-424b-aacb-b2f7c4ca792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:04.290738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:04.445737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:04.448032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:04.570714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd071e-e9ff-4a9c-b39f-63031778f5a9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:04.577794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:04.578411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:04.578688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:04.579001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:04.587537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:04.589132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee6de9-e2a4-46f7-84a9-6475846fa511 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:04.598595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 751/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:05.434451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3a12-fc5f-4b3a-93f1-dd48ea372ca6 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:05.435405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3a12-fc5f-4b3a-93f1-dd48ea372ca6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:05.435405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf3a12-fc5f-4b3a-93f1-dd48ea372ca6 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:05.437510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:47:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:05.443590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eeabd-9fa6-4ff8-8227-51017a15a417 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:05.444055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eeabd-9fa6-4ff8-8227-51017a15a417 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:05.444163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eeabd-9fa6-4ff8-8227-51017a15a417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:05.469348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e60eeabd-9fa6-4ff8-8227-51017a15a417 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:05.470537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e60eeabd-9fa6-4ff8-8227-51017a15a417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:05.478971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:05.639056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf3a12-fc5f-4b3a-93f1-dd48ea372ca6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:05.657104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf3a12-fc5f-4b3a-93f1-dd48ea372ca6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:05.662871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:05.663498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:05.663776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:05.664049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:05.673635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:05.675325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc50cbd-7d07-4a56-a49e-3058b5601707 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:05.685310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 764/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:06.429506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959dc8-59d3-449e-a8df-01fbf7f23055 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:06.429779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959dc8-59d3-449e-a8df-01fbf7f23055 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:06.430087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e959dc8-59d3-449e-a8df-01fbf7f23055 None None] PATH_INFO: `/v3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:06.431462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 753/1517] 10.222.0.57 () {54 vars in 925 bytes} [Sat Sep 3 03:47:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:47:06.438626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed738ea-6673-4179-a4b2-728fb61fc68a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:06.439181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed738ea-6673-4179-a4b2-728fb61fc68a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:06.439181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed738ea-6673-4179-a4b2-728fb61fc68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:06.504286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-bed738ea-6673-4179-a4b2-728fb61fc68a None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:06.505513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bed738ea-6673-4179-a4b2-728fb61fc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:06.523606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:47:06.581606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 765/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Sat Sep 3 03:47:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:10.389944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4f2a-cae4-4b32-aa2b-be1df6bd0503 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:10.390522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4f2a-cae4-4b32-aa2b-be1df6bd0503 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:10.390522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4f2a-cae4-4b32-aa2b-be1df6bd0503 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:10.391938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 754/1519] 10.222.0.25 () {54 vars in 961 bytes} [Sat Sep 3 03:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:10.397372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b6f4-dd6a-4a14-a391-02fc9dbbc941 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:10.397816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b6f4-dd6a-4a14-a391-02fc9dbbc941 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:10.397918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b6f4-dd6a-4a14-a391-02fc9dbbc941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:10.419122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d1f6b6f4-dd6a-4a14-a391-02fc9dbbc941 None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:10.420323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d1f6b6f4-dd6a-4a14-a391-02fc9dbbc941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:10.429480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 766/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Sep 3 03:47:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:10.437218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a4f2a-cae4-4b32-aa2b-be1df6bd0503 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:10.452649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a4f2a-cae4-4b32-aa2b-be1df6bd0503 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:10.457851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:10.458441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:10.458715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:10.459014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:10.467208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:10.468793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9e7c-ccf9-44ba-82dc-bd88f778d521 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:10.480407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 755/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:16.930235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311100-c660-408e-9a11-0b460d0dc7d5 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:16.930913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311100-c660-408e-9a11-0b460d0dc7d5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:16.930913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311100-c660-408e-9a11-0b460d0dc7d5 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:16.932191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:16.938259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dea05-d8e7-4fe0-ade6-8a12338293c2 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:16.938725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dea05-d8e7-4fe0-ade6-8a12338293c2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:16.938853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dea05-d8e7-4fe0-ade6-8a12338293c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:16.966448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d90dea05-d8e7-4fe0-ade6-8a12338293c2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:16.967693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d90dea05-d8e7-4fe0-ade6-8a12338293c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:16.976899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:17.128784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91311100-c660-408e-9a11-0b460d0dc7d5 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:17.144618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91311100-c660-408e-9a11-0b460d0dc7d5 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:17.150214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:17.150829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:17.151128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:17.151397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:17.159572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:17.161214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efa0ae-1d42-4562-a63c-c3656a924e4d service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:17.170766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 768/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:17.990790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef70c-b39b-4757-abc5-1ed8c0571dfe None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:17.991297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef70c-b39b-4757-abc5-1ed8c0571dfe None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:17.991409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef70c-b39b-4757-abc5-1ed8c0571dfe None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:17.993494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 03:47:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:18.000760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bc0fc-2bfa-4202-bbdf-f3daf0e6755a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:18.001245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bc0fc-2bfa-4202-bbdf-f3daf0e6755a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:18.001354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bc0fc-2bfa-4202-bbdf-f3daf0e6755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:18.021434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5d2bc0fc-2bfa-4202-bbdf-f3daf0e6755a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:18.022642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5d2bc0fc-2bfa-4202-bbdf-f3daf0e6755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:18.031123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 03:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:18.181951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeef70c-b39b-4757-abc5-1ed8c0571dfe None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:18.207199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeef70c-b39b-4757-abc5-1ed8c0571dfe None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:18.214162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:18.214824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:18.215123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:18.215391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:18.224169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:18.225809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e805a-f7fa-40bf-909a-a2ca26547809 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:18.236112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 758/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:22.036432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:22.037541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:22.037541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:22.039605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:47:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:22.048232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622394-73ec-4843-a8aa-2538042449dc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:22.048668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622394-73ec-4843-a8aa-2538042449dc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:22.048820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622394-73ec-4843-a8aa-2538042449dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:22.077448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-59622394-73ec-4843-a8aa-2538042449dc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:22.078672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-59622394-73ec-4843-a8aa-2538042449dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:22.087419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:22.375592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:22.378501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:22.461310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2926dd2-a30d-4701-807b-3d8a451c126e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:22.465093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:22.465697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:22.465973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:22.466237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:22.475785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:22.477200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a25b6-c38b-434e-8a97-46767d414f3b service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:22.486904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 771/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:24.374369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0c2-b114-4d60-b005-3391aa8c2be5 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:24.374995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0c2-b114-4d60-b005-3391aa8c2be5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:24.374995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0c2-b114-4d60-b005-3391aa8c2be5 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:24.376314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:24.382780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffd473-f3a9-4fe7-b779-719899a7a81d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:24.383459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffd473-f3a9-4fe7-b779-719899a7a81d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:24.383630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffd473-f3a9-4fe7-b779-719899a7a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:24.414484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-22ffd473-f3a9-4fe7-b779-719899a7a81d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:24.415851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-22ffd473-f3a9-4fe7-b779-719899a7a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:24.425658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:24.709147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa0c2-b114-4d60-b005-3391aa8c2be5 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:24.724324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa0c2-b114-4d60-b005-3391aa8c2be5 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:24.729547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:24.730344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:24.730678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:24.731021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:24.744207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:24.745642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13feea7-8ce9-44a6-8f37-09c1e1b06115 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:24.755388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 761/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:28.245778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:28.246304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:28.246304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:28.247669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:28.253285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e61e5-d0c6-4321-ada6-5a7906618d79 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:28.253689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e61e5-d0c6-4321-ada6-5a7906618d79 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:28.253935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e61e5-d0c6-4321-ada6-5a7906618d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:28.278082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-be3e61e5-d0c6-4321-ada6-5a7906618d79 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:28.279339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-be3e61e5-d0c6-4321-ada6-5a7906618d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:28.287917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:28.839155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:28.841284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:28.912761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:47:28.927420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f287-2d20-4364-8ac7-76c271db89fe None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:28.931886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:28.932511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:28.932789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:28.933054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:28.943302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:28.944771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91883c1b-e5ac-4e67-9947-efb7370f382b service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:28.954440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 774/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:30.880244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:30.881047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:30.881047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:30.882444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 03:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:47:30.888577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4488f6-3ecf-4c0e-9b37-ea88860f9365 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:30.889052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4488f6-3ecf-4c0e-9b37-ea88860f9365 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:30.889147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4488f6-3ecf-4c0e-9b37-ea88860f9365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:30.922573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cc4488f6-3ecf-4c0e-9b37-ea88860f9365 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:47:30.923826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cc4488f6-3ecf-4c0e-9b37-ea88860f9365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:30.934774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 03:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:47:31.484273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:47:31.487534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:47:31.497173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=84 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:47:31.504369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a40bbf-d55e-41c0-babc-d17245c72dc8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:47:31.508001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:47:31.508551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:47:31.508824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:47:31.509090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:47:31.518846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:47:31.520303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9340e-7353-4bfc-9d79-9c41e6c3e12a service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:47:31.530127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 764/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:47:40.915561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=11 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:47:41.525820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:53.537504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.538138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.538138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.566340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:53.568487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:53.582310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 776/1540] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:53.592322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.592940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.592940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.625038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:53.626290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:53.639313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 765/1541] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:53.645041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:53.660047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c256a4-d91a-4780-9ebd-d897fd786bdb None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:53.666338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:53.667408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.667916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.668402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.669291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:53.669823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:53.672136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5ca0-4ff9-4c1d-b35f-6ce652a1d883 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:53.690899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 777/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:53.700042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:53.712607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d305275-cd62-48c5-8268-467e312f83ee None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:53.719477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:53.720113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.720387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.720655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.721306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:53.723958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:53.773404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfa31-a9ff-46ce-9083-7af545b8d867 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.774138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfa31-a9ff-46ce-9083-7af545b8d867 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.774634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfa31-a9ff-46ce-9083-7af545b8d867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.784091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 766/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:53.789070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:53.808277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5bcdfa31-a9ff-46ce-9083-7af545b8d867 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:53.809992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5bcdfa31-a9ff-46ce-9083-7af545b8d867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:53.910388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 778/1544] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:48:53.921022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5c92-70cd-4564-9785-dcb0ddac34e5 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.921672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5c92-70cd-4564-9785-dcb0ddac34e5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.922161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5c92-70cd-4564-9785-dcb0ddac34e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.938169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d435c7-793d-4253-8111-3e1eb82efd8b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:53.953198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f6db5c92-70cd-4564-9785-dcb0ddac34e5 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:53.958831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:53.958831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.958831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.958831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:53.959302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:53.959302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:48:53.959302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:48:53.959302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:48:53.959302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:53.960501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f6db5c92-70cd-4564-9785-dcb0ddac34e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:53.969652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 779/1545] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:53.976362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2afc-fa51-4465-a364-4d7b349396ea None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:53.977021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2afc-fa51-4465-a364-4d7b349396ea None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:53.977515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af2afc-fa51-4465-a364-4d7b349396ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.010202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-07af2afc-fa51-4465-a364-4d7b349396ea None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.011918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-07af2afc-fa51-4465-a364-4d7b349396ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.018151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 767/1546] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:48:54.020966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.051112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 780/1547] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.053544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2851e-406f-473d-9eb1-ed6e3d9364a6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.053833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2851e-406f-473d-9eb1-ed6e3d9364a6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.054106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2851e-406f-473d-9eb1-ed6e3d9364a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.063541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99654d7-6d90-41f8-9d1d-3a97cd6668e3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.084523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.085149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.085423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.085691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.086347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.086650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.087800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:48:54.087800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:48:54.087800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:48:54.088704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.103213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7dc2851e-406f-473d-9eb1-ed6e3d9364a6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.104433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7dc2851e-406f-473d-9eb1-ed6e3d9364a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.113854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 768/1548] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.114675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.123742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 781/1549] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.125102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4cb4-13c4-4a0a-9d5f-e5d1d57d23b0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.125382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4cb4-13c4-4a0a-9d5f-e5d1d57d23b0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.125651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4cb4-13c4-4a0a-9d5f-e5d1d57d23b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.140546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08cf14-2c6d-4f66-81b1-250a15e06a2d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.150699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.151405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.151680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.151956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.152596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.154049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-222e4cb4-13c4-4a0a-9d5f-e5d1d57d23b0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.154187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.155655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-222e4cb4-13c4-4a0a-9d5f-e5d1d57d23b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.168480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 782/1550] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.170979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 769/1551] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:54.171637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c884-86dc-4418-b9db-2f306f9811d7 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.171906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c884-86dc-4418-b9db-2f306f9811d7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.172022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.172222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c884-86dc-4418-b9db-2f306f9811d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.185457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759f432-1c46-4fd2-a3f6-42e6d67318f3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.191530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.192174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.192450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.192724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.193351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.193630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a156c884-86dc-4418-b9db-2f306f9811d7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.194864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a156c884-86dc-4418-b9db-2f306f9811d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.195313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a05b1-4886-4701-9b36-0856091c30cf admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.202745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 783/1552] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.203947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ae48-86d8-4552-bb7f-a887329c4e23 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.204231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ae48-86d8-4552-bb7f-a887329c4e23 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.204494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ae48-86d8-4552-bb7f-a887329c4e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.223786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 770/1553] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.225248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.225502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.225941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.227021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-6896ae48-86d8-4552-bb7f-a887329c4e23 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.228241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6896ae48-86d8-4552-bb7f-a887329c4e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.246919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.248144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.299049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 784/1554] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.300139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.300424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.301094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.314322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 771/1555] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.315694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.320494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.322158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.325683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09a30a-df40-4522-9a08-092ad0437a26 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.330858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 785/1556] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.331023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.331135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.331221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.331564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.331832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.332674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.332970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.334584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e791d83-a944-4058-89f0-470ce2355c51 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.336144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:54.346760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d648dc-c49a-46de-b44f-45bad437fa83 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.349512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 772/1557] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.350835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.351120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.351394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.351590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.352234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.352519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.352794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.353464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.355141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8b1d6-b19b-4dd7-85e7-49b9958ae2b5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.366489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 786/1558] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:54.367721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.368010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.368281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.371377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.372605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.380487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 773/1559] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.383376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.387739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.389257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.396135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25ae0a-5b04-4900-8aa4-f424c6e797ab None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.399054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 787/1560] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.400063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.401128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.401737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.402021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.402280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.402943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.404985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4bd00-f130-4d06-8b0c-df4d2e8314f0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.410345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b1c6c-6577-434d-853f-9254cfe7a611 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.415001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.415666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.415968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.416884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.418090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.418752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.421121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ede12-3526-43da-a46e-059fd5980945 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.435571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 774/1561] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.444101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.444277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 788/1562] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.444390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.444390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.444554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.444554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.444554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.462405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.464234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.465583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.467401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.476874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 789/1563] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.481190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.481288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 775/1564] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.481288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.486485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:54.490668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c5c66-3775-4088-80a8-f42300af7d0c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.491943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b18b2c-5a96-4d71-8aae-812995fc5048 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.495816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.496467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.496676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.496944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.497095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.497576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.497792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.498079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.498341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] PATH_INFO: `/v3/projects/798ac7907fc047889ade5c052b2140fb/users/d1801a169eec4a388c4438f93ddc2779/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.498615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:48:54.498615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:48:54.498615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:48:54.499840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d523aa2-8ef0-401e-9524-9b4cb67b7d69 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.513548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798ac7907fc047889ade5c052b2140fb, user_id=d1801a169eec4a388c4438f93ddc2779, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.516287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.528962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 790/1565] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.530224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.530513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.530788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.531311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 776/1566] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:54 2022] PUT /identity/v3/projects/798ac7907fc047889ade5c052b2140fb/users/d1801a169eec4a388c4438f93ddc2779/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:54.532261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.560770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.562015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.595423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd59399-0f7c-44ea-9f57-d70461f3cd59 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.626138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 791/1567] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.627408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.636599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d826-bb87-403c-9cd1-039d9c740ced None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.641889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.642486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.642767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.643063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.643709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.645351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.655283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 792/1568] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:54.656075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.657212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.657926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.658268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.658637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.659318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.659619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.661364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.666544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c574c9-8a2a-4e5e-a8dd-31b57bb3815d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.672475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.673065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.673337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.680555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 777/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.680759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.680867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.680970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.680970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.686254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb6732-74d5-4b9d-bd36-ecfe55f171cc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.691286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.691996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.692269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.692545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.693182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.693480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.695091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.704351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 793/1570] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.705186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.718517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 778/1571] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.718517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.724014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348020b-d95d-4ee7-810e-34fa4081ddd9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.749362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:54.770644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edab20-8e0c-402b-9b8b-87ee4851ab9f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.775696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.776301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.776577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.776844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.777481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.777778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.779562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7ab4b-c74a-405c-941b-13053d28c497 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.784008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.784618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.784893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.785163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.785845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.787813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.792734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 794/1572] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.794070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.794357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.794629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.813343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 779/1573] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.813343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.820900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe375059-bf6d-4921-b81d-01d35b1c9eb2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.825642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.826300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.826595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.826874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.827622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:48:54.827951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.828628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] MFA Rules not processed for user `d1801a169eec4a388c4438f93ddc2779`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:54.829566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.829943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:54.841862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 780/1574] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:48:54.842671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.857721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13b2b-89b8-41c1-9833-1fbf9ee897f5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.862828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.863617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.863952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.864297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.864966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.866580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.867210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:54.877380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 781/1575] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:54.878176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.888454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ba464-cd49-4c9c-b4aa-1c82530cc5d1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.893019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.893597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.893867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.894133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.894940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.897577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.904040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 795/1576] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:54.908791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.919638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc09df-55ce-45b2-bf28-4caa83b93ed0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:54.925568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:54.926657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:54.927159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:54.927462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:54.928302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:54.930437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:54.934057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 782/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.934057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.958191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 796/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:54.958191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:54.967826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6ea94-4a28-4a32-abff-bc2c39dd2f60 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.004870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c641ee-2e2c-4d0d-ba61-21023085912a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.009702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.010323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.010601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.010903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.011544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.013474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.025753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.026371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.026646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.026995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.027679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.030268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.040943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 783/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.042048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.065984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 797/1580] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:54 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.065984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.108414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.113857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b6a1-cd03-4d0d-b975-cbec5dba5ed6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.128768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c36c-68d6-4d8d-a30f-2a4e8c3b5d97 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.134062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.135089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.135553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.136031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.137091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.139825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.169992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 798/1581] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.170826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.175289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.175962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.176237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.176505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.177156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.178988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a8eec-11bf-4324-bf1a-7e67d6d40c6c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.189802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 784/1582] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.191157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dde84-37f7-4dca-9648-35a5b4650e19 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.191444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dde84-37f7-4dca-9648-35a5b4650e19 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.191743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dde84-37f7-4dca-9648-35a5b4650e19 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.193051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 785/1583] 10.222.0.25 () {54 vars in 962 bytes} [Sat Sep 3 03:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:48:55.193786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dde84-37f7-4dca-9648-35a5b4650e19 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.204347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.242476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77662e-36ca-4f62-a316-7420c980721a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.248275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.249179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.249458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.249743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] PATH_INFO: `/v3/projects/12347de4d2e5465b9a00004084fcb85c/users/58361634962b4b7884d51be90b683b42/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.260510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dde84-37f7-4dca-9648-35a5b4650e19 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.265096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.265444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12347de4d2e5465b9a00004084fcb85c, user_id=58361634962b4b7884d51be90b683b42, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.265706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.265985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.266287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.266483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:48:55.266483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:48:55.266483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:48:55.266951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.268511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.268865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.283777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 799/1584] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:55 2022] PUT /identity/v3/projects/12347de4d2e5465b9a00004084fcb85c/users/58361634962b4b7884d51be90b683b42/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:55.284689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.294744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 786/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.298554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.308131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed159e-6711-443c-8785-47aec9c6e356 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.346489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214b82c-3c3e-4327-9499-bf6f13ebb576 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.351059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.351673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.351949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.352216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.352856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.354461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.363230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.363836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.364110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.364374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.364482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 800/1586] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.367930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.368074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.368074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.374467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d852c-6f72-400b-890a-d5bc1792bc15 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.379486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.380418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.380705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.381082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.381796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.384289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.389902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 787/1587] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.390685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.403221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d45165-87cd-4af1-874f-b10ec95f7515 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.406426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 801/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.407328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.410952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.411548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.411824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.412097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.412719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.414634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8038c6-ddb2-4ba0-b802-232e860dece3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.417232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957c55f-1130-4a0b-9105-830ba8e4c373 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.421753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.422336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.422614 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.422919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.423576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.425147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9f95e-a2d9-44bd-8123-608e4a70110b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.432317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.436135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 802/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.437344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.437678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.437955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.440048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 788/1590] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:55.441492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.442140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.442939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.461346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:55.462533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:55.470748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 789/1591] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:55.471532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.477239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:55.478662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:55.480978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01edd26-4ad7-472d-9632-515437da9089 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.485887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.486521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.486765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.487077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.487696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.489339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.502009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 790/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.502009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.510730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be998-b251-4e08-b842-57732ae7acb7 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.516915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.517954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.518394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.518859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.519671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.521383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.532619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 791/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.533406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.535688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.544937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cb67-9201-433a-b83e-d3ec35ceeddf admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.549776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.550096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 803/1594] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:55.550364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.550638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.550839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.551003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.551596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.553183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.561265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715cf6d-ec67-40e3-90cd-4e160334b6c2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.563613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 792/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.564233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.568030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.568633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.568914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.569185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.569828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.571464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.573589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15519-6b9c-4a2a-94a8-179bf026dc12 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.578051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.578680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.579008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.579277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] PATH_INFO: `/v3/projects/ac9e77355ba040bda4f2186753df7b24/users/1dde7fad075145c89f726ca85c6c00ba/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.584891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 804/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.584891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.592885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb781f1-8ba0-41b0-866c-c75ec25c386e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.594592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9e77355ba040bda4f2186753df7b24, user_id=1dde7fad075145c89f726ca85c6c00ba, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.597332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.597966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.598403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.598685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.598993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.599615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.601170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.611051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 793/1597] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:55 2022] PUT /identity/v3/projects/ac9e77355ba040bda4f2186753df7b24/users/1dde7fad075145c89f726ca85c6c00ba/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:55.614921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.615063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 805/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.615063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.624334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558198ac-4da2-410e-acd7-b1e6ea1723d7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.668984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca61e6-a711-46e4-969e-0a7967ab65f0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.673378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.674000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.674283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.674560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.681530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.682245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.682531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.682822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.683472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.685071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.695211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 806/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.695941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.705405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997945f-2293-4826-b1e6-724d0deee6ce admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.710365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.711029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.711299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.711568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] PATH_INFO: `/v3/projects/f3077cfc588042e0a1becefe1363385c/users/ba00478a6ef74115b83726083026f574/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.726933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3077cfc588042e0a1becefe1363385c, user_id=ba00478a6ef74115b83726083026f574, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.729575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.738372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.740345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.743857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 807/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:55 2022] PUT /identity/v3/projects/f3077cfc588042e0a1becefe1363385c/users/ba00478a6ef74115b83726083026f574/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:55.744655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.754084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 794/1601] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:55.755196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.765285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d530c1-77a7-43ce-a908-b146bca72b3e service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.811681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.822414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4032-8cc4-4d19-8a9b-6a031b210054 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.827103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.828085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.828437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.828953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.829485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.829754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.830234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.830608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.830969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.831750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.842894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 795/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.843908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.854791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aac3f-2492-4ab1-874b-4a93b5cd5fa1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.859920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.860593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.860938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.861381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.862121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.863966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce2b81-30bb-4056-8bb2-de035aa114fe admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.878466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 796/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.878466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.878466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.878466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.884694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:55.902701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:55.904005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:55.910925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.912137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 797/1604] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:55.912922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.913045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.922861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292f83a-e9ad-4497-8fa2-2d2a17affbbc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.925068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 808/1605] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:55.926087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.935634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.935850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.936133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.936565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.938003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.940259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcbfb3-0f2f-4a28-9f76-b957ae73aa7c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.942789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3969d-897d-4de2-b3eb-ff6aaff086ea service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:55.949953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:55.950983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.951403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.951762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] PATH_INFO: `/v3/projects/ae92edf3ac0c4e0c9fff79714b62a4ab/users/b3663594897d45479afee85eb5333d80/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.959051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 798/1606] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:55.960618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:55.961053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:55.961488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:55.970284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae92edf3ac0c4e0c9fff79714b62a4ab, user_id=b3663594897d45479afee85eb5333d80, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:55.973866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:55.995586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 809/1607] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:55 2022] PUT /identity/v3/projects/ae92edf3ac0c4e0c9fff79714b62a4ab/users/b3663594897d45479afee85eb5333d80/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:55.996549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:55.999339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.000814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.081133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329829-95f5-4331-9960-fe00bcf84750 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.083515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 799/1608] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.084792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.087195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.088161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.088535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.088907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] PATH_INFO: `/v3/projects/5404143f5ac54f778c649bee74e51c13/users/f4b3f2822726453a9f39e0b2bb76b5ef/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.106212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5404143f5ac54f778c649bee74e51c13, user_id=f4b3f2822726453a9f39e0b2bb76b5ef, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.106793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e397a-1163-4169-ac09-9ab40f3ceb2d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.110282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.120970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.121903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.122401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.122926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] PATH_INFO: `/v3/projects/4d80eadc5cc34322b85201c50b3c6552/users/31a8c62bd81f4c1f863bd952b6976276/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.132861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 810/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:55 2022] PUT /identity/v3/projects/5404143f5ac54f778c649bee74e51c13/users/f4b3f2822726453a9f39e0b2bb76b5ef/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:56.135970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.146063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.154500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d80eadc5cc34322b85201c50b3c6552, user_id=31a8c62bd81f4c1f863bd952b6976276, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.157404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.181217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 800/1610] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:56 2022] PUT /identity/v3/projects/4d80eadc5cc34322b85201c50b3c6552/users/31a8c62bd81f4c1f863bd952b6976276/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:56.182271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.234595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96268f-4d29-49c9-8afe-72b1b092a7eb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.263665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829b43-a650-460d-9e59-c37ab704ca05 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.269138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.270218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.270494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.271015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] PATH_INFO: `/v3/projects/968cc9376f884f67b20b91547211fffd/users/a959dec6c0c944aea231b65612892705/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.287289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968cc9376f884f67b20b91547211fffd, user_id=a959dec6c0c944aea231b65612892705, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.290078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f6bcc-c866-439a-ad60-133cca09fd70 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.297648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.297847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.298149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.298410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.304601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 801/1611] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:56 2022] PUT /identity/v3/projects/968cc9376f884f67b20b91547211fffd/users/a959dec6c0c944aea231b65612892705/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:56.305950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fdc74-b5ec-406d-b41b-3a35f6649699 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.306211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fdc74-b5ec-406d-b41b-3a35f6649699 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.306487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fdc74-b5ec-406d-b41b-3a35f6649699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.308805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.310305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.316147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:56.322099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 811/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:56.323356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.332666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa8f3-8e71-425b-bf1a-ecc64834115d service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.336258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:56.340765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-152fdc74-b5ec-406d-b41b-3a35f6649699 None None] MFA Rules not processed for user `b3663594897d45479afee85eb5333d80`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.343105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-152fdc74-b5ec-406d-b41b-3a35f6649699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.388045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.388643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.388920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.390056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.427943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 802/1613] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.429263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.429548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.429824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.461277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] MFA Rules not processed for user `f4b3f2822726453a9f39e0b2bb76b5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.462510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.463206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.464882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178a55-bf02-47c5-b147-e09570a7651b service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.476347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 812/1614] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:56.477762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b05cf-3906-4209-9a6f-a0f886047a5b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.478048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b05cf-3906-4209-9a6f-a0f886047a5b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.478416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b05cf-3906-4209-9a6f-a0f886047a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.501783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9e8b05cf-3906-4209-9a6f-a0f886047a5b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.501783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9e8b05cf-3906-4209-9a6f-a0f886047a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.513208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 813/1615] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.514787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1386-8dfa-4405-b4c8-d40e2dac43f3 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.515104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1386-8dfa-4405-b4c8-d40e2dac43f3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.515589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1386-8dfa-4405-b4c8-d40e2dac43f3 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.516943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 814/1616] 10.222.0.25 () {54 vars in 912 bytes} [Sat Sep 3 03:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:48:56.518066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.518311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.518583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.536054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 803/1617] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.540580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.542644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.554064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 815/1618] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.554974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.555097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.569879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a23c-dd05-497f-9686-414a55487967 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.580500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.582403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.582626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.583850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.586242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a88b17-0194-48c0-b6ea-a9b6e7732619 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.594837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.595725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.596194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.596499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.597640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.600096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.602876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.604513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28bcaf-e876-4661-811f-0201efcab123 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.616591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 816/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:56.618132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376962-acc4-472f-9e37-ad197d7fdaee None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.618475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376962-acc4-472f-9e37-ad197d7fdaee None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.618713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376962-acc4-472f-9e37-ad197d7fdaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.641062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 804/1620] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:56.642140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.646254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-7e376962-acc4-472f-9e37-ad197d7fdaee None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:56.647754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7e376962-acc4-472f-9e37-ad197d7fdaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:56.746566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 817/1621] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:56.754293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee04e7-da1d-4fc0-9901-7e0adb8e566d None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.758578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee04e7-da1d-4fc0-9901-7e0adb8e566d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.759143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee04e7-da1d-4fc0-9901-7e0adb8e566d None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.762938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 818/1622] 10.222.0.25 () {54 vars in 962 bytes} [Sat Sep 3 03:48:56 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:48:56.764560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee04e7-da1d-4fc0-9901-7e0adb8e566d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.764777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c539d4-aac0-47ae-a7b4-41a6cca529a0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.785068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.785915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.786221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.786490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.816341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:56.843065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:56.901164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee04e7-da1d-4fc0-9901-7e0adb8e566d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:56.910542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:56.911532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.912205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.913644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.914697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:56.916212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:56.918526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.921110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0e2d-f95f-4202-94cb-9d3302d32f92 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:56.944119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 805/1623] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:56.945709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:56.968446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 819/1624] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:48:56.971542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:56.972894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:56.973569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.025743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:57.029733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:57.051532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 820/1625] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Sep 3 03:48:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:57.052601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.072951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bbf56-2000-4d6d-bf6d-d1682cc4815b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.074850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f766153-d89b-4beb-a73d-cb379c55bc85 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.082208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.083338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.084360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.085217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.085709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.087158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.087627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.088308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.090023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.092366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.104976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 821/1626] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:57.106036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.112518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.114443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.121082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8658-fb03-42cb-b167-88904355a508 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.130906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.131598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.131814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 806/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:57.132854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.133191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.133482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.145373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c23c4-68ad-4a3e-a0c1-720961757958 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.152081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.152700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.153109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.153494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] PATH_INFO: `/v3/projects/c00ba11d3b4041179ed29f4906eefe0d/users/73a48f855e8e40e8bf9d1f0b500ace2f/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.188725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00ba11d3b4041179ed29f4906eefe0d, user_id=73a48f855e8e40e8bf9d1f0b500ace2f, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.193353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.223160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 807/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:57 2022] PUT /identity/v3/projects/c00ba11d3b4041179ed29f4906eefe0d/users/73a48f855e8e40e8bf9d1f0b500ace2f/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:57.227251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.259110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.261391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.272663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 822/1629] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:57.381895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c70e-868f-4270-aa62-61809a24756d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.394421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.395289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.395659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.396066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.396836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.398542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571331-a14d-4886-a623-428977acc7db admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.419265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 808/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:48:57.423649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.438842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724617-856e-47d6-9b38-9cc22f186dfe service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.455644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.456325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.456602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.456877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] PATH_INFO: `/v3/projects/c00ba11d3b4041179ed29f4906eefe0d/users/73a48f855e8e40e8bf9d1f0b500ace2f/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.462703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00ba11d3b4041179ed29f4906eefe0d, user_id=73a48f855e8e40e8bf9d1f0b500ace2f, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.467668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:57.509448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 823/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:48:57 2022] PUT /identity/v3/projects/c00ba11d3b4041179ed29f4906eefe0d/users/73a48f855e8e40e8bf9d1f0b500ace2f/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:57.509997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee127-8d14-4dc9-8248-7b89288a27cd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.510345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee127-8d14-4dc9-8248-7b89288a27cd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.510676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cee127-8d14-4dc9-8248-7b89288a27cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.576697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-60cee127-8d14-4dc9-8248-7b89288a27cd None None] MFA Rules not processed for user `73a48f855e8e40e8bf9d1f0b500ace2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:57.578015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-60cee127-8d14-4dc9-8248-7b89288a27cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:57.623257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:57.725106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 809/1632] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:57.737889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:57.899406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc061-01d0-42f1-b188-163d7d0699b0 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.900019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc061-01d0-42f1-b188-163d7d0699b0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.900519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc061-01d0-42f1-b188-163d7d0699b0 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.902970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 810/1633] 10.222.0.25 () {54 vars in 912 bytes} [Sat Sep 3 03:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:48:57.914504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.915013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.915447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.930004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bd19-6b53-445e-a45f-ca04da9d1da7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:57.935353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:57.936388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:57.936804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:57.937176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:57.949717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:57.951325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:57.958220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:57.973625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:58.021816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 824/1634] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:58.060717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 811/1635] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:48:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:48:58.131888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:58.141472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4652-449e-4d3e-9190-53870f31f148 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:58.150278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:58.151032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:58.151312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:58.151576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:58.168889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:58.170402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435405ff-e343-4679-810b-eda82d4975fb service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:58.214505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 825/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:48:59.045041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:59.208783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cbfaa-0dad-45de-b61d-963eb52ab85f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:59.228457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:59.229132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:59.229415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:59.229690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] PATH_INFO: `/v3/users/b3663594897d45479afee85eb5333d80` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:59.232269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3663594897d45479afee85eb5333d80)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:59.233929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:59.356826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3663594897d45479afee85eb5333d80'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:48:59.358513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3663594897d45479afee85eb5333d80'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:48:59.370556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:59.389233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3663594897d45479afee85eb5333d80'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:48:59.405060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3663594897d45479afee85eb5333d80'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:48:59.405935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3663594897d45479afee85eb5333d80'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:48:59.451418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 812/1637] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:48:59 2022] DELETE /identity/v3/users/b3663594897d45479afee85eb5333d80 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:48:59.456731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:59.457294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:59.457385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:59.469223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:48:59.483950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:48:59.485192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:59.569915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 826/1638] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:48:59.588754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:48:59.591292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:48:59.687120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2a6da-5ca2-45c9-b42b-9e05e29f32c6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:48:59.699153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:48:59.699935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:48:59.700803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:48:59.701092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:48:59.713006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:48:59.715052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:48:59.732487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 813/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:01.106273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:01.108251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:01.127947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a555d1-cb4e-4b14-85d3-97d5733e0353 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:01.167218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:01.168237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:01.168655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:01.169414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] PATH_INFO: `/v3/projects/ae92edf3ac0c4e0c9fff79714b62a4ab` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:01.171791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae92edf3ac0c4e0c9fff79714b62a4ab)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:01.174245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:01.327119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:01.328127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:01.335774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae92edf3ac0c4e0c9fff79714b62a4ab', 'request_id': 'req-798375b3-87a2-4123-ae55-52b3678ef0d6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.346244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b105f-21ea-446f-83ae-f0e45c5cb85a service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:01.363780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae92edf3ac0c4e0c9fff79714b62a4ab', 'request_id': 'req-798375b3-87a2-4123-ae55-52b3678ef0d6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.377801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae92edf3ac0c4e0c9fff79714b62a4ab', 'request_id': 'req-798375b3-87a2-4123-ae55-52b3678ef0d6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.419794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] The token cache is being invalidate because project ae92edf3ac0c4e0c9fff79714b62a4ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:01.419989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.421900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.427447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 827/1640] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:01 2022] DELETE /identity/v3/projects/ae92edf3ac0c4e0c9fff79714b62a4ab => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:01.437689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:01.442795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:01.520770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:01.521503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:01.525671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:01.526046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] PATH_INFO: `/v3/users/d1801a169eec4a388c4438f93ddc2779` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:01.528582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1801a169eec4a388c4438f93ddc2779)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:01.530855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:01.626858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798375b3-87a2-4123-ae55-52b3678ef0d6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:01.633744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:01.634620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:01.635323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:01.635488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] PATH_INFO: `/v3/users/73a48f855e8e40e8bf9d1f0b500ace2f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:01.637380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a48f855e8e40e8bf9d1f0b500ace2f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:01.638430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: WARNING py.warnings [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:49:01.638430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: warnings.warn(msg) Sep 03 03:49:01.638430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: Sep 03 03:49:01.639691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:01.644769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1801a169eec4a388c4438f93ddc2779'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.645676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1801a169eec4a388c4438f93ddc2779'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.686182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1801a169eec4a388c4438f93ddc2779'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.697935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1801a169eec4a388c4438f93ddc2779'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.698971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-02d5ce63-3b71-4a80-b16b-0e210f60daa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1801a169eec4a388c4438f93ddc2779'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.702287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:01.716687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:01.724922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 814/1641] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:01 2022] DELETE /identity/v3/users/d1801a169eec4a388c4438f93ddc2779 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:01.726390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:01.726683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:01.727010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:01.749472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:01.750825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a48f855e8e40e8bf9d1f0b500ace2f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.750953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:01.783779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a48f855e8e40e8bf9d1f0b500ace2f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.796866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a48f855e8e40e8bf9d1f0b500ace2f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.798237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a48f855e8e40e8bf9d1f0b500ace2f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.798898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a48f855e8e40e8bf9d1f0b500ace2f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:01.816092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 828/1642] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:01 2022] DELETE /identity/v3/users/73a48f855e8e40e8bf9d1f0b500ace2f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:01.840227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 815/1643] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:01.856741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:01.858469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:01.972901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e7268-5b76-4bad-95ea-bcb3cf991a31 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:01.980129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:01.981545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:01.983233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:01.983857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:02.075633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:02.078325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ed02-0204-44ac-af3b-c75f75f5c382 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:02.092305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 829/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:49:02.938454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:02.940383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:02.961096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af6865-3b6f-4ffd-8cf9-622887eb2d7a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:02.972802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:02.973473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:02.973746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:02.974206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] PATH_INFO: `/v3/projects/c00ba11d3b4041179ed29f4906eefe0d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:02.982477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00ba11d3b4041179ed29f4906eefe0d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:02.983745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: WARNING py.warnings [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 03:49:02.983745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: warnings.warn(msg) Sep 03 03:49:02.983745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: Sep 03 03:49:02.985693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:03.121400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00ba11d3b4041179ed29f4906eefe0d', 'request_id': 'req-43911b5f-1253-4239-ac63-4a33de2c2c1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:03.150145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00ba11d3b4041179ed29f4906eefe0d', 'request_id': 'req-43911b5f-1253-4239-ac63-4a33de2c2c1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:03.167832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00ba11d3b4041179ed29f4906eefe0d', 'request_id': 'req-43911b5f-1253-4239-ac63-4a33de2c2c1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:03.210141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] The token cache is being invalidate because project c00ba11d3b4041179ed29f4906eefe0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:03.210458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:03.215734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-43911b5f-1253-4239-ac63-4a33de2c2c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:03.218003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 816/1645] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:02 2022] DELETE /identity/v3/projects/c00ba11d3b4041179ed29f4906eefe0d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:03.224458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.224856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.224856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.266481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:03.267852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:03.383464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:03.412772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 830/1646] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:03.421964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.422230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.422504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.458315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:03.459587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:03.483706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 817/1647] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:03.488166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.500346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de062c-eca5-4f6c-86be-4d2c34cd6a02 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:03.513085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:03.513732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.514010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.514283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.515379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:03.515689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:03.517595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:03.548789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 831/1648] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:03.552799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.565756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78856d6-e114-4625-abe2-b6fe8e819b9b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:03.579797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:03.580599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.580888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.581159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.581818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:03.589135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:03.653268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 818/1649] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:03 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:03.657238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.668297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172762c-5bd5-455c-91d3-d22c3696d45d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:03.804151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:03.804478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.804851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.805220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.805979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:03.808465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:03.875093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 832/1650] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:03 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:03.881301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.905876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03444f44-a759-4b31-a398-8d34f0e5db48 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:03.932586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:03.933618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:03.933898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:03.934175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:03.935376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:03.937759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:03.944572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.950002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:03.982680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 819/1651] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:03.986612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:03.987047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c020fa8-30dd-4690-a0c0-46caff8683e5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:03.999316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.000595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.001181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.001588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] PATH_INFO: `/v3/projects/798ac7907fc047889ade5c052b2140fb` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.005214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798ac7907fc047889ade5c052b2140fb)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.008797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.011858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e561-b14e-46ce-a7a5-715039b699fc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.025965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.027724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.028164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.028577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.031683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.043274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.092228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 820/1652] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:04.093190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.120685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa2976-188e-49a1-a8f6-3436c51de5ba admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.145498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.146185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.146470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.146738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] PATH_INFO: `/v3/projects/8a0959f256b54b5ea338f8fbcd5db8e6/users/6551f98c79ce487998cc7f730eb7af0a/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.188157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0959f256b54b5ea338f8fbcd5db8e6, user_id=6551f98c79ce487998cc7f730eb7af0a, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.191105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020b2a3-b6c0-42c3-978b-600f7e070b6a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.192649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798ac7907fc047889ade5c052b2140fb', 'request_id': 'req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:04.208022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798ac7907fc047889ade5c052b2140fb', 'request_id': 'req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:04.213303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 821/1653] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:04 2022] PUT /identity/v3/projects/8a0959f256b54b5ea338f8fbcd5db8e6/users/6551f98c79ce487998cc7f730eb7af0a/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:04.217810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798ac7907fc047889ade5c052b2140fb', 'request_id': 'req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:04.224137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.224558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.224927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.238087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] The token cache is being invalidate because project 798ac7907fc047889ade5c052b2140fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:04.243776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:04.246459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:04.247594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0f6cfb3a-3f7f-4472-8c2e-11e74d7c0974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:04.249865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 833/1654] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:03 2022] DELETE /identity/v3/projects/798ac7907fc047889ade5c052b2140fb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:04.261149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a65f-693c-4a46-82d8-e460273e93ee None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.262435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a65f-693c-4a46-82d8-e460273e93ee None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.265122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a65f-693c-4a46-82d8-e460273e93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.276460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] MFA Rules not processed for user `6551f98c79ce487998cc7f730eb7af0a`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:04.278875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:04.307870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9663a65f-693c-4a46-82d8-e460273e93ee None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:04.309608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9663a65f-693c-4a46-82d8-e460273e93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:04.411769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 822/1655] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:04.416219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.417764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:04.456112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:04.480862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 834/1656] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:04.489381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11001-8449-4574-993c-e370415197d1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.490142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11001-8449-4574-993c-e370415197d1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.490631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11001-8449-4574-993c-e370415197d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.518566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a5e11001-8449-4574-993c-e370415197d1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:04.520389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a5e11001-8449-4574-993c-e370415197d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:04.531039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 835/1657] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:04.535499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e11001-8449-4574-993c-e370415197d1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.548069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e11001-8449-4574-993c-e370415197d1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.553455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.554567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.555171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.555881 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.556649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:04.557202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.559584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.577811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 836/1658] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:49:04.581863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.604150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea863-e0b9-44b9-9829-48b6795ca692 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.610459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5560b-d80a-4bc9-8cac-efc4f684bd3c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.613139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.619432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.619809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.620149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.621325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.623008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.629675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.630567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.635138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.636185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.637308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.646249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.697457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 837/1659] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:04.701665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.724004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 823/1660] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/projects => generated 387 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:04.726600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.834752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96076492-cff9-4f5b-9aaf-0cf90ad3419f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.869659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6202b-eb9c-4046-9d23-eb56e850d64d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.876444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.877104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.877385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.877663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.878365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.884882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.921946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 824/1661] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:04.925950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.941082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aac5c-c7f9-4bc0-942f-4577015aa1b9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:04.950278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.951082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.951266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.951542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.952201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.953916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.954115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:04.954828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:04.955109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:04.955382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:04.956540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:04.958047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:04.960437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:04.989064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 825/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:04.994355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 838/1663] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:04 2022] POST /identity/v3/users => generated 284 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:04.998019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:04.998580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.010383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2881e1-30b2-4a71-b943-c4359d2bb00f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.016253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.017002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.017415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.017802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.019232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.020760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.036814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 839/1664] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:05.038658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.043709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280d5c-1928-49c5-beea-5e2330cf75e4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.056258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.057414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.057908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.058412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] PATH_INFO: `/v3/projects/5ac4985ddb274751b330e5915e0914ff/users/994a00321bf442589007f2f5003268c6/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.074651 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cbb0-dbce-451f-bb8f-562e35729ce5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.092939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.093593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.093874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.096190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.099619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.100529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac4985ddb274751b330e5915e0914ff, user_id=994a00321bf442589007f2f5003268c6, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.103862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.109289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.139953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 826/1665] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:04 2022] PUT /identity/v3/projects/5ac4985ddb274751b330e5915e0914ff/users/994a00321bf442589007f2f5003268c6/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:05.143906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.154353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 840/1666] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:05.156428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.183167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d7c17-ab1d-4a00-b2a3-835b00d175f0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.222497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:05.264648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.265587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.265930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.266314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] PATH_INFO: `/v3/projects/83d4854db25142a6b0be8a11bf8dabb8/users/7502170bc3684b6f9648432349a54e5f/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.294198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d4854db25142a6b0be8a11bf8dabb8, user_id=7502170bc3684b6f9648432349a54e5f, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.297287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccfeb3-b554-4deb-8d53-412711e8260c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.302275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445f1f-5874-4239-acc8-2d07a5dd04f2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.320219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 841/1667] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:05 2022] PUT /identity/v3/projects/83d4854db25142a6b0be8a11bf8dabb8/users/7502170bc3684b6f9648432349a54e5f/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:05.330518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.331272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.331557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.331831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.331993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.332286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.332551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.332832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.334199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.357574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 827/1668] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:05.358542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.384006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0619ab-7fd6-4a8b-8af8-36c86f7a473c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.406108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] MFA Rules not processed for user `7502170bc3684b6f9648432349a54e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:05.407669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:05.524294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.524511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.524776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.525059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] PATH_INFO: `/v3/projects/5ac4985ddb274751b330e5915e0914ff/users/994a00321bf442589007f2f5003268c6/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.527195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac4985ddb274751b330e5915e0914ff, user_id=994a00321bf442589007f2f5003268c6, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.530242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0690b-14d6-422c-a583-3d7a0cccaf27 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.562049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 828/1669] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:05 2022] PUT /identity/v3/projects/5ac4985ddb274751b330e5915e0914ff/users/994a00321bf442589007f2f5003268c6/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:05.572333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.572621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.572899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.595031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 842/1670] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:05.608370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.609834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:05.651455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] MFA Rules not processed for user `994a00321bf442589007f2f5003268c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:05.652708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:05.754073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9479bd-1ea0-478b-8a4e-2bb7733f8e63 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.799269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:05.858470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.859196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.859476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.859796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.861252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 829/1671] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:05.872053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:05.928327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:05.929752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:05.953946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d347-99b3-438f-a7a5-d73649cbd020 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:05.965155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.966110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:05.966624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:05.966745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:05.967108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:05.967316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:05.988736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 843/1672] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:49:05.993809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:05.995332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971fc97-9466-4c5a-a6ec-476b7e34e658 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:06.021154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 830/1673] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:49:06.256374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:06.462926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ffcd2-84b4-4f24-bc0b-f43d2e7652d2 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:06.484998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:06.485814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:06.486168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:06.486518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] PATH_INFO: `/v3/users/6551f98c79ce487998cc7f730eb7af0a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:06.487984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6551f98c79ce487998cc7f730eb7af0a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:06.490517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:06.605746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6551f98c79ce487998cc7f730eb7af0a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:06.652263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6551f98c79ce487998cc7f730eb7af0a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:06.665600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6551f98c79ce487998cc7f730eb7af0a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:06.666478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6551f98c79ce487998cc7f730eb7af0a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:06.667379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6551f98c79ce487998cc7f730eb7af0a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:06.684608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 844/1674] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:06 2022] DELETE /identity/v3/users/6551f98c79ce487998cc7f730eb7af0a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:06.689254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:06.689800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:06.690540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:06.745077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:06.756568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:06.918493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 831/1675] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:06.958616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:06.960481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:07.050125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:07.099463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214c9-9e5b-4302-b9ad-5aff48fd6290 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:07.111226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:07.112040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:07.112417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:07.112792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:07.142310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:07.143959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00455bd-07b4-486e-a751-8201841779ab service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:07.179341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 845/1676] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:07.701101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:07.703637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:07.742083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448afd02-98d6-4333-ab4e-293989171da4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:07.755078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:07.755765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:07.756041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:07.756312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] PATH_INFO: `/v3/users/f4b3f2822726453a9f39e0b2bb76b5ef` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:07.761813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b3f2822726453a9f39e0b2bb76b5ef)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:07.766833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:07.953230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b3f2822726453a9f39e0b2bb76b5ef'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:07.954885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b3f2822726453a9f39e0b2bb76b5ef'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:07.989499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b3f2822726453a9f39e0b2bb76b5ef'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:08.006948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b3f2822726453a9f39e0b2bb76b5ef'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:08.009667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b3f2822726453a9f39e0b2bb76b5ef'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:08.049457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 832/1677] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:07 2022] DELETE /identity/v3/users/f4b3f2822726453a9f39e0b2bb76b5ef => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:08.058940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:08.059518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:08.059518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:08.097562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:08.098839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:08.225124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 846/1678] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:08.342039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:08.344363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:08.482532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:08.510288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b06012-f356-4b5c-9a48-c648503f70b8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:08.517853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:08.519560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:08.521722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:08.523264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:08.549182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:08.563011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:08.581639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 833/1679] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:09.125758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:09.129322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:09.150526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e52d6a-2f91-462b-9fbf-47f0c97dad49 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:09.164057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:09.165030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:09.167568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:09.168163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] PATH_INFO: `/v3/projects/8a0959f256b54b5ea338f8fbcd5db8e6` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:09.170845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0959f256b54b5ea338f8fbcd5db8e6)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:09.176426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:09.280096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0959f256b54b5ea338f8fbcd5db8e6', 'request_id': 'req-bdfdc464-d754-404b-887b-2702e108ab24'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.301541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0959f256b54b5ea338f8fbcd5db8e6', 'request_id': 'req-bdfdc464-d754-404b-887b-2702e108ab24'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.326617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0959f256b54b5ea338f8fbcd5db8e6', 'request_id': 'req-bdfdc464-d754-404b-887b-2702e108ab24'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.369067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] The token cache is being invalidate because project 8a0959f256b54b5ea338f8fbcd5db8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:09.369380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.371185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.374968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 847/1680] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:09 2022] DELETE /identity/v3/projects/8a0959f256b54b5ea338f8fbcd5db8e6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:09.379110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:09.394770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a04f77-4628-436d-b1c8-d59e7822214c service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:09.541457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:09.542180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:09.543447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:09.543780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] PATH_INFO: `/v3/users/994a00321bf442589007f2f5003268c6` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:09.548593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994a00321bf442589007f2f5003268c6)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:09.551164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:09.674533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994a00321bf442589007f2f5003268c6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.675324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994a00321bf442589007f2f5003268c6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.698944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994a00321bf442589007f2f5003268c6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.713217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994a00321bf442589007f2f5003268c6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.719328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c1c5c21-a955-446e-9b70-dba2bde12947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994a00321bf442589007f2f5003268c6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:09.752221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 834/1681] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:09 2022] DELETE /identity/v3/users/994a00321bf442589007f2f5003268c6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:10.711448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:10.715851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:10.742785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdc464-d754-404b-887b-2702e108ab24 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:10.954644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:10.978347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:10.979181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:10.979572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:10.979948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] PATH_INFO: `/v3/projects/5404143f5ac54f778c649bee74e51c13` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:10.983991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5404143f5ac54f778c649bee74e51c13)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:10.986458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:11.095065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5404143f5ac54f778c649bee74e51c13', 'request_id': 'req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:11.111155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5404143f5ac54f778c649bee74e51c13', 'request_id': 'req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:11.133430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5404143f5ac54f778c649bee74e51c13', 'request_id': 'req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:11.148785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] The token cache is being invalidate because project 5404143f5ac54f778c649bee74e51c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:11.149234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:11.150438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d4a4dce4-32f6-41bf-a3c0-e24a1c441b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:11.152536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 848/1682] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:10 2022] DELETE /identity/v3/projects/5404143f5ac54f778c649bee74e51c13 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:11.157366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.157909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.158258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.201582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:11.204040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:11.382092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 835/1683] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:11.393753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.394204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.394656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.445437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:11.449732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:11.476456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 849/1684] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:11.480781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:11.494989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e606f-d968-4b35-9e03-2e582aaa5eab None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:11.510586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:11.510756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.511006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.511290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.511934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:11.512232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:11.514137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:11.523490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:11.550650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 836/1685] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:11.552311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:11.570570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60627e9b-17a6-4ce7-b877-801d30457ba1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:11.581929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:11.583819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.584112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.584386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.585522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:11.591344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:11.693824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:11.694882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 850/1686] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:11 2022] POST /identity/v3/projects => generated 403 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:11.720197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a1022-afb5-4643-b8ac-d9857855ed3d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:11.887905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:11.888795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.889261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.889877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.890722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:11.892917 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:11.933220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 837/1687] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:11 2022] POST /identity/v3/users => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:11.937336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:11.953585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1c66-b833-4341-9634-cc4409f31ef6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:11.974416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:11.975870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:11.976214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:11.976495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:11.978515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:11.984564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:12.020958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 851/1688] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:12.021906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:12.042087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02347f7e-638f-4607-bcf1-60c0986b02bc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:12.067431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:12.068073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.068373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.070956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.072033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:12.073698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc3b4-089f-4e25-a2a8-5c8552f0d127 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:12.112207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:12.112498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 838/1689] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:12.146473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f30-fff4-418e-aefb-eb01fb45cae7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:12.161719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:12.163477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.163976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.164608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] PATH_INFO: `/v3/projects/1bc5bb0684ed4dca98e39e502d8ca69d/users/54cb4ee25a7f47ee9be87fd924dbcaa7/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.199133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc5bb0684ed4dca98e39e502d8ca69d, user_id=54cb4ee25a7f47ee9be87fd924dbcaa7, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:12.203841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1642c1-f9b7-4e50-b058-1c2504483e60 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:12.229803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 852/1690] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:12 2022] PUT /identity/v3/projects/1bc5bb0684ed4dca98e39e502d8ca69d/users/54cb4ee25a7f47ee9be87fd924dbcaa7/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:12.244553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842a02-4719-4030-b041-0d9feba65802 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.245041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842a02-4719-4030-b041-0d9feba65802 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.245121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842a02-4719-4030-b041-0d9feba65802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.254031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.258146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.258457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.342164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-88842a02-4719-4030-b041-0d9feba65802 None None] MFA Rules not processed for user `54cb4ee25a7f47ee9be87fd924dbcaa7`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:12.343894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-88842a02-4719-4030-b041-0d9feba65802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:12.375877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] MFA Rules not processed for user `58361634962b4b7884d51be90b683b42`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:12.385887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:12.499057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 839/1691] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:12.500073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88842a02-4719-4030-b041-0d9feba65802 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:12.504017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-88842a02-4719-4030-b041-0d9feba65802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:12.608365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 853/1692] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:12.608365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:12.647875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:12.764757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88842a02-4719-4030-b041-0d9feba65802 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:12.779674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:12.780516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.780938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.786529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] PATH_INFO: `/v3/projects/5ac4985ddb274751b330e5915e0914ff` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.789371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac4985ddb274751b330e5915e0914ff)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:12.791923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:12.810784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:12.836669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2c5c4-6513-4944-93e9-457e63889b50 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:12.856204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:12.856850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:12.857440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:12.857721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:12.858377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:12.872877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:12.960013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 854/1693] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:12 2022] POST /identity/v3/projects => generated 403 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:12.963417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:12.963543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac4985ddb274751b330e5915e0914ff', 'request_id': 'req-f3480df9-3b5a-453f-b194-34abb76787c5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:12.968237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:12.979841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac4985ddb274751b330e5915e0914ff', 'request_id': 'req-f3480df9-3b5a-453f-b194-34abb76787c5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:13.001440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac4985ddb274751b330e5915e0914ff', 'request_id': 'req-f3480df9-3b5a-453f-b194-34abb76787c5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:13.055116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] The token cache is being invalidate because project 5ac4985ddb274751b330e5915e0914ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:13.055423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:13.063688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f3480df9-3b5a-453f-b194-34abb76787c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:13.073684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 840/1694] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:12 2022] DELETE /identity/v3/projects/5ac4985ddb274751b330e5915e0914ff => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:13.084480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329ee8-40e6-4e67-8d44-42154bb440f0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.086686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329ee8-40e6-4e67-8d44-42154bb440f0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.087172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329ee8-40e6-4e67-8d44-42154bb440f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.187026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3212-c405-4fa4-966c-f6c6c7b30ca1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:13.202175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:13.203641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.204256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.204779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.207098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.211514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fc329ee8-40e6-4e67-8d44-42154bb440f0 None None] MFA Rules not processed for user `1dde7fad075145c89f726ca85c6c00ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:13.214202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fc329ee8-40e6-4e67-8d44-42154bb440f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.399352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:13.401199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:13.443094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 855/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:13.443983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:13.458507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.460550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 841/1696] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:13.462112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.462647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.463045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.490617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:13.492121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.622489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:13.722659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffbb3f-0c63-4289-9663-a458ec8bb19f service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:13.745639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 842/1697] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:13.745639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:13.746289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.749056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:13.749868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.750501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.750931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.751917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:13.754246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f13ec-5c7f-419c-a55c-e34282d7e213 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:13.800950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 856/1698] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:13 2022] POST /identity/v3/users => generated 308 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:13.801637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a40b9-5c71-4b45-a8d5-e8ae91c98f80 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:13.802725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.803203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.803671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.818982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:13.819593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.819930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.820200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.840001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:13.841792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:13.859957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 843/1699] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:13.861239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:13.884916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebf492-0431-4fc7-8c5d-9e77cd626a10 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:13.900024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:13.900701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.900972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.901252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.902375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:13.903838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e8e4b-134e-47ed-954d-6ce480e487d5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:13.913878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:13.916926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:13.918902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:13.931447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 844/1700] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:13.933003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:13.941902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:13.942784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:13.958008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 857/1701] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:13.962417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.001970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2447bc8-04cf-4307-9218-e473709f872a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.016305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:14.017251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.019051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.019549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] PATH_INFO: `/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b/users/96464f0a7b4a4936bbaca87ca189ec2a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.035570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] MFA Rules not processed for user `ba00478a6ef74115b83726083026f574`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:14.036991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:14.073105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a3a71ac0c44ea48a1aebc1b0e0aa5b, user_id=96464f0a7b4a4936bbaca87ca189ec2a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:14.082098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:14.125464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 858/1702] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:13 2022] PUT /identity/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b/users/96464f0a7b4a4936bbaca87ca189ec2a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:14.134175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.259852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 845/1703] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:14.265332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.312020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76765212-96c9-4136-9b11-73d1e5c7f3b5 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.324440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af01cc-d2ef-4ef6-84b8-e04e3849850b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.327616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:14.328800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.329227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.332390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.333426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:14.338857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba8d6f-36ed-4fd3-a86f-e7c9be95d387 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:14.346476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:14.347813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.348215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.348586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.349321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:14.349690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:14.352236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:14.372319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 846/1704] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:14.373941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d17e6-56d0-4a4a-80b4-703091215cd8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.374562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d17e6-56d0-4a4a-80b4-703091215cd8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.375302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d17e6-56d0-4a4a-80b4-703091215cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.375633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 859/1705] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:14.376306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.400639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df3c-452b-4e2f-800b-450caf80c3bb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.411608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:14.412504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.412977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.413818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] PATH_INFO: `/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b/users/96464f0a7b4a4936bbaca87ca189ec2a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.416238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ca9d17e6-56d0-4a4a-80b4-703091215cd8 None None] MFA Rules not processed for user `ba00478a6ef74115b83726083026f574`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:14.418947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a3a71ac0c44ea48a1aebc1b0e0aa5b, user_id=96464f0a7b4a4936bbaca87ca189ec2a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:14.420503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ca9d17e6-56d0-4a4a-80b4-703091215cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:14.424362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:14.454953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 860/1706] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:14 2022] PUT /identity/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b/users/96464f0a7b4a4936bbaca87ca189ec2a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:14.456319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.556944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 847/1707] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:14.560224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.563429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.563841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.626950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891444-ddb0-4a05-95e5-c6c48b94e473 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.645339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:14.646136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:14.646518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:14.646866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:14.647832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:14.649802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:14.667849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] MFA Rules not processed for user `96464f0a7b4a4936bbaca87ca189ec2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:14.669140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:14.768279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:14.781311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 861/1708] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:14 2022] POST /identity/v3/projects => generated 387 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:14.785742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.791440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:14.933010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 848/1709] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:14.935207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:14.963676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cb2df-4352-4be6-a54b-775129d73b70 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:14.989502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:15.016413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccc5fe-52a6-49cb-bf3d-d7343785d141 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:15.035266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.035915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.036196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.036517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.161360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.166326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.167220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.167651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.168713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.169748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.173592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.178446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.195948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 862/1710] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:15.197342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:15.201361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:15.248033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 849/1711] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:14 2022] POST /identity/v3/users => generated 294 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:15.252780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:15.294785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7a66-aaaf-4a79-87ee-ab836fb67534 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:15.321973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.325656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.326037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.326459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.327164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.340814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.404126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 850/1712] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:15.411880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:15.431726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd50ac-ad48-4ec4-bc1c-3a3c48560c99 service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:15.438121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41281ad-8417-4cd4-81f2-cddc946a9aea admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:15.450119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.450861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.451281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.451397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.463015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.466469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.466758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.467114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.468535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.471240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.497634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 851/1713] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:15.499501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:15.519755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:15.521455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f343d-3090-4dae-b71f-51f95010632f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:15.535408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:15.536612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.537429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.538818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] PATH_INFO: `/v3/projects/032a9710741b4b268a44b8b17b209c7d/users/d79b846b67e94b39b3a450833c095082/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.562017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.565080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1443086-44cd-461e-b895-76cea5499c08 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.569652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032a9710741b4b268a44b8b17b209c7d, user_id=d79b846b67e94b39b3a450833c095082, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:15.581014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611104-8f13-4e36-812f-1d8b2ec54dc4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:15.594632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 863/1714] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:15 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:15.595986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.596271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.596590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.615466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 852/1715] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:15 2022] PUT /identity/v3/projects/032a9710741b4b268a44b8b17b209c7d/users/d79b846b67e94b39b3a450833c095082/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:15.629925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacabd1-6dd4-421a-bc12-bbba98566ce8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.632815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacabd1-6dd4-421a-bc12-bbba98566ce8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.633254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacabd1-6dd4-421a-bc12-bbba98566ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.680249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] MFA Rules not processed for user `a959dec6c0c944aea231b65612892705`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:15.682573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:15.725818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9aacabd1-6dd4-421a-bc12-bbba98566ce8 None None] MFA Rules not processed for user `d79b846b67e94b39b3a450833c095082`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:15.727630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9aacabd1-6dd4-421a-bc12-bbba98566ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:15.897447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 864/1716] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:15.898447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:15.922386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 853/1717] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:15.924076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec237-a425-4a33-8671-993d706d663c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:15.924191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec237-a425-4a33-8671-993d706d663c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:15.924438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec237-a425-4a33-8671-993d706d663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:15.955116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-98cec237-a425-4a33-8671-993d706d663c None None] MFA Rules not processed for user `a959dec6c0c944aea231b65612892705`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:15.957713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-98cec237-a425-4a33-8671-993d706d663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:15.968798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 854/1718] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:15.970130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec237-a425-4a33-8671-993d706d663c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:16.048221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:16.091610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e908-5890-4d84-a88c-351a4b9de52b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:16.102306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec237-a425-4a33-8671-993d706d663c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:16.106531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:16.107543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.109976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.110275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] PATH_INFO: `/v3/users/54cb4ee25a7f47ee9be87fd924dbcaa7` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.111638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cb4ee25a7f47ee9be87fd924dbcaa7)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:16.113669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:16.115906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:16.118046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.118046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.118046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.118046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:16.120375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:16.201380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 855/1719] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:15 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:16.203242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:16.225342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cb4ee25a7f47ee9be87fd924dbcaa7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:16.265934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cb4ee25a7f47ee9be87fd924dbcaa7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:16.287225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cb4ee25a7f47ee9be87fd924dbcaa7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:16.289649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cb4ee25a7f47ee9be87fd924dbcaa7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:16.291671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cb4ee25a7f47ee9be87fd924dbcaa7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:16.302679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:16.324249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 865/1720] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:15 2022] DELETE /identity/v3/users/54cb4ee25a7f47ee9be87fd924dbcaa7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:16.326715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:16.331044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:16.339239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c3892-8ba2-4f9c-86f8-421de1a8c039 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:16.521279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:16.521932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.522210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.522477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.527100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:16.580097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713663d-a26e-4047-bd8b-00c1ed1fa108 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:16.605862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:16.606520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.607625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.608243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.608967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:16.615195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b8271-7efa-4f3d-aa13-aa31a87497bb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:16.671028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 866/1721] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:16 2022] POST /identity/v3/users => generated 294 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:16.675760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064112-7295-46e5-b725-ffe95bec8404 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.676071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064112-7295-46e5-b725-ffe95bec8404 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.677047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064112-7295-46e5-b725-ffe95bec8404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.720542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1a064112-7295-46e5-b725-ffe95bec8404 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:16.725493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1a064112-7295-46e5-b725-ffe95bec8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:16.738886 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:16.745142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:16.749337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:16.763655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 867/1722] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:16.765640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7f17-7f74-4f23-837d-ecbfd9d109ec None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.771380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7f17-7f74-4f23-837d-ecbfd9d109ec None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.771698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7f17-7f74-4f23-837d-ecbfd9d109ec None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.774553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 856/1723] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:16.775688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:16.785377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 868/1724] 10.222.0.25 () {54 vars in 912 bytes} [Sat Sep 3 03:49:16 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:49:16.791922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.792215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.792983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.800207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaa47-a9aa-4f2d-826b-5b3432b1ee25 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:16.813073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:16.816161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:16.816161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:16.816929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:16.817401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:16.821838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:16.850187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 857/1725] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:16.850187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:16.856441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:16.914865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:16.916063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:17.033166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85b4e-a221-41ab-992b-fafafceec025 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.047488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.048367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.048781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.049220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.074504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.076235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.096064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 858/1726] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:17.097220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.102253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 869/1727] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:17.116450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085de9d-8736-4cd7-8829-070f9da21d1b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.129924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.130921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.131388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.131837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.132682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.134676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.147070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 859/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:17.151029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.179731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d57abd-9417-41bc-81f1-c59cadc8da60 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.186421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.187195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.187477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.187752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] PATH_INFO: `/v3/projects/ffb0ee08212d49ca98190a0a74a401e3/users/fc20a48c71da490abe7362928812085c/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.219273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb0ee08212d49ca98190a0a74a401e3, user_id=fc20a48c71da490abe7362928812085c, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.223750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.224327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510f28a-4341-46b4-a732-93e9a02c804b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.226325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:17.257415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95e0f-44ef-400a-9b65-77e629fbfe77 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.259151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 870/1729] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:17 2022] PUT /identity/v3/projects/ffb0ee08212d49ca98190a0a74a401e3/users/fc20a48c71da490abe7362928812085c/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:17.262172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.273839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.274383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.333839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] MFA Rules not processed for user `fc20a48c71da490abe7362928812085c`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:17.340903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:17.393908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.394871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.395220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.395730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.415081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.416993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f879ca-3f47-4f16-ae70-3a766f2258e3 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.437415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 860/1730] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:17.438861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.439149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.439415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.451379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:17.508768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] MFA Rules not processed for user `31a8c62bd81f4c1f863bd952b6976276`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:17.510207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:17.516329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 871/1731] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:17.517512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.668268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5dbc4-4e72-4fff-815d-c280324ab7c9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.674320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.675249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.676003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.676500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.677153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 861/1732] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:17.677266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.679930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.732857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 872/1733] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:17 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:17.733804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.735514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:17.736475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.777920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbbdab-06ab-4758-adcc-1e23e8ba7204 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.800582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:17.919948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175344a-34a5-4cf1-b4a3-4cfc4f917cd7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:17.937615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.938293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.938621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.939051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.940004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.942541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:17.983495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:17.984155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:17.984441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:17.985323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] PATH_INFO: `/v3/users/7502170bc3684b6f9648432349a54e5f` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:17.986472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 873/1734] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:17 2022] POST /identity/v3/users => generated 299 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:17.987726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:17.989058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7502170bc3684b6f9648432349a54e5f)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:17.991854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:18.019139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79320ae-dc9b-4252-8418-286ddc1cbc14 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:18.032706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:18.036478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.037153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.040168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.040168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:18.061391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:18.100989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 874/1735] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:18.112136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:18.114965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7502170bc3684b6f9648432349a54e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:18.115704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7502170bc3684b6f9648432349a54e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:18.164724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c3841-09c3-4613-9b39-cd128ed3bcb1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:18.174264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7502170bc3684b6f9648432349a54e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:18.186203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:18.188983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.189460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.190102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] PATH_INFO: `/v3/projects/bf78a06119f34740a3b52d17a4e37002/users/d8d66db4c659443b8a8e374d3ab1a47c/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.197227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7502170bc3684b6f9648432349a54e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:18.198101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8d0ca52-19b8-49ca-ae00-a550410e38d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7502170bc3684b6f9648432349a54e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:18.241354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf78a06119f34740a3b52d17a4e37002, user_id=d8d66db4c659443b8a8e374d3ab1a47c, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:18.241758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 862/1736] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:17 2022] DELETE /identity/v3/users/7502170bc3684b6f9648432349a54e5f => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:18.245415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.245706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.245985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.248917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:18.277404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 875/1737] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:18 2022] PUT /identity/v3/projects/bf78a06119f34740a3b52d17a4e37002/users/d8d66db4c659443b8a8e374d3ab1a47c/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:18.281629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:18.283437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:18.328180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:18.329326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:18.346489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:18.464541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a86fd-1479-4361-9ead-3627455ecc44 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:18.487782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:18.489104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.489404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.489683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.490361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:18.492037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:18.514662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 876/1738] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:18.515857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:18.531110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 863/1739] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:18.531925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6a07e-b32c-475f-837f-7fc0adae9a1e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:18.561286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:18.563473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:18.572688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:18.574511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.580716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.581119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] PATH_INFO: `/v3/projects/bf78a06119f34740a3b52d17a4e37002/users/d8d66db4c659443b8a8e374d3ab1a47c/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.585350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf78a06119f34740a3b52d17a4e37002, user_id=d8d66db4c659443b8a8e374d3ab1a47c, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:18.588720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49559d-44cf-4f33-8542-f0768c6e6e00 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:18.642220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 877/1740] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:18 2022] PUT /identity/v3/projects/bf78a06119f34740a3b52d17a4e37002/users/d8d66db4c659443b8a8e374d3ab1a47c/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:18.642220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315418-c140-43f4-a2c1-9c7a673c8ac1 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.642220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315418-c140-43f4-a2c1-9c7a673c8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.642220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315418-c140-43f4-a2c1-9c7a673c8ac1 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.648791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 878/1741] 10.222.0.25 () {54 vars in 912 bytes} [Sat Sep 3 03:49:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:49:18.650277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad27a-c0f6-4b24-ba63-8d5f59863f66 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.650561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad27a-c0f6-4b24-ba63-8d5f59863f66 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.650870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad27a-c0f6-4b24-ba63-8d5f59863f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.667428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:18.706029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2efad27a-c0f6-4b24-ba63-8d5f59863f66 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:18.706543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2efad27a-c0f6-4b24-ba63-8d5f59863f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:18.732674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0778569-5f21-4396-8a98-d52adcab4316 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:18.868690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 879/1742] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:18.870473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.870853 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:18.871314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.871314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.871486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:18.871486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:18.871755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:18.946993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] MFA Rules not processed for user `d8d66db4c659443b8a8e374d3ab1a47c`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:18.948459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.034287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.037024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.039162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 880/1743] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:19.039162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.039162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.064267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 864/1744] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:19.065216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.074862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.084091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51d4aa-327f-442d-9a0a-dd38034b2d7f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:19.105087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:19.105928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.108289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.112639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.131850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.135415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.147570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 881/1745] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:19.149389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.151116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.193608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7ba62-aea7-4a1b-bed0-d5a8348eaa7b service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:19.203924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:19.205391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.206008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.207065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] PATH_INFO: `/v3/projects/1bc5bb0684ed4dca98e39e502d8ca69d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.209493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc5bb0684ed4dca98e39e502d8ca69d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.210059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23291d-a45b-4d8c-962f-25b9e98e3084 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:19.213499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.229608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:19.230298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.230582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.230884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.246639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.248287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.271784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 865/1746] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:19.272865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.279736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.319133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:19.411010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc5bb0684ed4dca98e39e502d8ca69d', 'request_id': 'req-881be777-ba09-4c7e-8317-e983c80f3c48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:19.437408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc5bb0684ed4dca98e39e502d8ca69d', 'request_id': 'req-881be777-ba09-4c7e-8317-e983c80f3c48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:19.438432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:19.467840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc5bb0684ed4dca98e39e502d8ca69d', 'request_id': 'req-881be777-ba09-4c7e-8317-e983c80f3c48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:19.500764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] The token cache is being invalidate because project 1bc5bb0684ed4dca98e39e502d8ca69d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:19.501351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:19.502470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-881be777-ba09-4c7e-8317-e983c80f3c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:19.507461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 882/1747] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:19 2022] DELETE /identity/v3/projects/1bc5bb0684ed4dca98e39e502d8ca69d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:19.507461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.507953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.508338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.532009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459432fd-6825-4c37-a21c-0578e03cbf3b service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:19.564515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] MFA Rules not processed for user `58361634962b4b7884d51be90b683b42`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:19.584442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.716715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:19.717642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.718102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.718553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.725186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.777020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 883/1748] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:19.777020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.777020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.832719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.834515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.845529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 866/1749] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:19.846696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:19.848312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:19.923449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d31a2-a325-409c-904f-871d1b5322ae service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:19.928934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:19.929642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:19.930022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:19.930395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] PATH_INFO: `/v3/users/96464f0a7b4a4936bbaca87ca189ec2a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:19.932124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96464f0a7b4a4936bbaca87ca189ec2a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:19.934623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:19.951022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:20.037698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96464f0a7b4a4936bbaca87ca189ec2a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.041424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96464f0a7b4a4936bbaca87ca189ec2a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.088828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96464f0a7b4a4936bbaca87ca189ec2a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.113543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96464f0a7b4a4936bbaca87ca189ec2a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.115816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96464f0a7b4a4936bbaca87ca189ec2a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.141728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325c2a0-35b0-4456-9863-612b0826150f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:20.158543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 867/1750] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:19 2022] DELETE /identity/v3/users/96464f0a7b4a4936bbaca87ca189ec2a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:20.160345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:20.162524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:20.173465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:20.174039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:20.174327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:20.174605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:20.182138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:20.245281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:20.247597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea931b-0e69-4ca0-8686-0d5f95adf5a1 service placement] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:20.307221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 884/1751] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:20.392675 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee958-d6f6-4c09-b2a6-a0b8b6daf651 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:20.411016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:20.412575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:20.412854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:20.413230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] PATH_INFO: `/v3/projects/83d4854db25142a6b0be8a11bf8dabb8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:20.414541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d4854db25142a6b0be8a11bf8dabb8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:20.416610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:20.564602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:20.564900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:20.565281 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:20.612039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d4854db25142a6b0be8a11bf8dabb8', 'request_id': 'req-1145acc0-1210-4108-972a-7c7a8e4f4f4b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.641938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] MFA Rules not processed for user `1dde7fad075145c89f726ca85c6c00ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:20.642626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d4854db25142a6b0be8a11bf8dabb8', 'request_id': 'req-1145acc0-1210-4108-972a-7c7a8e4f4f4b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.646641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:20.659938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d4854db25142a6b0be8a11bf8dabb8', 'request_id': 'req-1145acc0-1210-4108-972a-7c7a8e4f4f4b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.701401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] The token cache is being invalidate because project 83d4854db25142a6b0be8a11bf8dabb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:20.703601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.725842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1145acc0-1210-4108-972a-7c7a8e4f4f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:20.729760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 868/1752] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:20 2022] DELETE /identity/v3/projects/83d4854db25142a6b0be8a11bf8dabb8 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:20.736499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88acb8-0159-488b-81df-a08978772281 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:20.737092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88acb8-0159-488b-81df-a08978772281 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:20.737542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88acb8-0159-488b-81df-a08978772281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:20.780559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:20.799565 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3c88acb8-0159-488b-81df-a08978772281 None None] MFA Rules not processed for user `ba00478a6ef74115b83726083026f574`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:20.801017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3c88acb8-0159-488b-81df-a08978772281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:20.839713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 885/1753] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:20.840814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:20.845079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:20.861326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:21.001713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02368689-46a6-42a5-bbf9-3b639510d116 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:21.007384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:21.008707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.009200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.009678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.012267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:21.013753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 869/1754] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:21.015231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94cee4-1dfb-481b-a48e-838d0c622718 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.015463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94cee4-1dfb-481b-a48e-838d0c622718 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.015742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94cee4-1dfb-481b-a48e-838d0c622718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.062074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1f94cee4-1dfb-481b-a48e-838d0c622718 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:21.063370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1f94cee4-1dfb-481b-a48e-838d0c622718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:21.248433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 870/1755] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:21.255075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.255753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.255753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.259572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:21.261760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:21.311064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:21.312536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:21.344915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 871/1756] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:21.346099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:21.352398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 886/1757] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:21.401553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96b36e-370c-4abb-85f2-4290014e061b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:21.436780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:21.443623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.444023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.444545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.445437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:21.446372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:21.451674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:21.504206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:21.505317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 872/1758] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:21.549076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7494c-4821-422c-81f4-bf09dd6f1b76 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:21.584712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:21.590898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.594215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.594482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.595213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:21.604377 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:21.665321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 887/1759] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:21 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:21.669572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:21.682872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a395-8183-4646-915e-f37b4251916f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:21.923589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:21.924848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:21.932223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:21.932689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:21.936383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:21.940379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.007130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 873/1760] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:21 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:22.007732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.020988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a715-b4ce-4f16-98f0-fd6571e023a7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:22.028544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:22.029570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.030056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.030484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.031326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:22.033315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.053084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.062061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 888/1761] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:22.066518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dd9ee-74ad-4003-a860-3e6f64c99953 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:22.072286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:22.073308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.073734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.074161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.074996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:22.076837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.115225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 874/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:22.116709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.134224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded45aa-58db-4823-b849-121b685d8702 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:22.151364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:22.152533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.155287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.158854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] PATH_INFO: `/v3/projects/59cdca9d9a0c4036ad29202a90a35e8e/users/a3644258235e4eb9840a74014a8e803a/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.202300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.204454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:22.218351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=59cdca9d9a0c4036ad29202a90a35e8e, user_id=a3644258235e4eb9840a74014a8e803a, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:22.228624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba993a78-67f2-4242-bc13-bdd7fa2aa0df admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.258268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a4030-cdee-489a-bfdd-d7b0043ae607 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:22.273194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 889/1763] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:22 2022] PUT /identity/v3/projects/59cdca9d9a0c4036ad29202a90a35e8e/users/a3644258235e4eb9840a74014a8e803a/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:22.283708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.283994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.284314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.370278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:22.401209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] MFA Rules not processed for user `a3644258235e4eb9840a74014a8e803a`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:22.402572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:22.497295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:22.498154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.498624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.499237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] PATH_INFO: `/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.503090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a3a71ac0c44ea48a1aebc1b0e0aa5b)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:22.507757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.563024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:22.660483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a3a71ac0c44ea48a1aebc1b0e0aa5b', 'request_id': 'req-2d2773e9-6214-441d-91a0-0e526275ec17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:22.672363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 890/1764] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:22.673120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.681599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a3a71ac0c44ea48a1aebc1b0e0aa5b', 'request_id': 'req-2d2773e9-6214-441d-91a0-0e526275ec17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:22.686156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3cffc-edf8-4836-bb34-adca7bd1d17a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:22.698584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a3a71ac0c44ea48a1aebc1b0e0aa5b', 'request_id': 'req-2d2773e9-6214-441d-91a0-0e526275ec17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:22.732692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] The token cache is being invalidate because project 82a3a71ac0c44ea48a1aebc1b0e0aa5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:22.733167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:22.735617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2d2773e9-6214-441d-91a0-0e526275ec17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:22.739890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 875/1765] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:22 2022] DELETE /identity/v3/projects/82a3a71ac0c44ea48a1aebc1b0e0aa5b => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:22.761616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced84-b8e8-41b7-b984-b242b6ec490e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.761616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced84-b8e8-41b7-b984-b242b6ec490e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.761616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced84-b8e8-41b7-b984-b242b6ec490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.804134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-70fced84-b8e8-41b7-b984-b242b6ec490e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:22.810150 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-70fced84-b8e8-41b7-b984-b242b6ec490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:22.845601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:22.846597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.847053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.847513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:22.850405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:22.865516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:22.925571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 891/1766] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:22.926618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:22.928225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:22.960096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 876/1767] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:22.961698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:22.961985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:22.962269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.015936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:23.018692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:23.036172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 877/1768] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:23.040719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.044748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d058f-0956-4b47-81dc-ebbbdad81d82 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.059436 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.063039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.065046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.065477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.066363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.066981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe2ba8-97c1-4a5e-aa38-8330c5051fad None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.075186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.084880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.087649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.088101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.088519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.095355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:23.095955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.099052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.140807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 892/1769] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:22 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:23.142021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.142871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 878/1770] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:49:23.145378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.168058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f286-c2a5-4ea5-a5c4-31f3bf58ef4a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.170043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4c192-f418-48c6-8db4-88bbd59cb1bc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.180353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.181234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.182136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.182644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.183424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.184225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.185057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.185483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.186024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.187522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.190923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.191523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.261536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 893/1771] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:23.280050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.302909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6f250-aad3-492e-82f6-c8a126a77d26 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.315548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.316214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 879/1772] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:23 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:23.317242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.320687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.321963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.322511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] PATH_INFO: `/v3/projects/d4c7a26c654f48a6ae03590c861df9e2/users/3d2748de3dcd41ddaed16a6b1937934a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.364211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c7a26c654f48a6ae03590c861df9e2, user_id=3d2748de3dcd41ddaed16a6b1937934a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.367797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.369805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:23.392517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 894/1773] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:23 2022] PUT /identity/v3/projects/d4c7a26c654f48a6ae03590c861df9e2/users/3d2748de3dcd41ddaed16a6b1937934a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:23.396874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.435907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:23.524970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d82d-b507-4d08-9b96-03ed4960ddfc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.625571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9a32-fbe4-441a-bdb6-2049a1ec5f22 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.637716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.639470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.639765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.640046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.640695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.645447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.676234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 895/1774] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:23.677478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.696741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.697384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.697667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.701625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d38b76-b385-4161-9c18-9013e5ded8b2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.702714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.703647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.705628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.716595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.717334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.717618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.718080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] PATH_INFO: `/v3/projects/d4c7a26c654f48a6ae03590c861df9e2/users/3d2748de3dcd41ddaed16a6b1937934a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.724143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c7a26c654f48a6ae03590c861df9e2, user_id=3d2748de3dcd41ddaed16a6b1937934a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.727326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8106e-565a-4e88-b4d7-ad8157795ed8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:23.757164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 880/1775] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:23 2022] POST /identity/v3/users => generated 291 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:23.759015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 896/1776] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:23 2022] PUT /identity/v3/projects/d4c7a26c654f48a6ae03590c861df9e2/users/3d2748de3dcd41ddaed16a6b1937934a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:23.761145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:23.770736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.771861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.771861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.792644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e34ff7-f13f-4e1b-9d55-dfcb43f09225 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:23.865994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] MFA Rules not processed for user `3d2748de3dcd41ddaed16a6b1937934a`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:23.876999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:23.981802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:23.982790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:23.983095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:23.984490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:23.985174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:23.988070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:24.003787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 881/1777] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:24.009293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:24.012291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 897/1778] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:24.022067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:24.024199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:24.033330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf104d-c441-453b-8a7a-71211dbc52b3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:24.040357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:24.041165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:24.041549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:24.041931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:24.043470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:24.046554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:24.072462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 882/1779] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:24.081532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:24.104622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aa351-4547-44ae-b14b-5592a34f01d1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:24.116246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:24.117293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:24.117830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:24.119595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] PATH_INFO: `/v3/projects/520e52fb9c0749c99fe04fcc65ee3559/users/173a41312eb940d2ab3e6b37106de975/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:24.142724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520e52fb9c0749c99fe04fcc65ee3559, user_id=173a41312eb940d2ab3e6b37106de975, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:24.147109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e80b-ad87-4ebb-86b5-8dbf0bf54012 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:24.191602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 883/1780] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:24 2022] PUT /identity/v3/projects/520e52fb9c0749c99fe04fcc65ee3559/users/173a41312eb940d2ab3e6b37106de975/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:24.196689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:24.196978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:24.197249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:24.198658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30abf5-16c9-4f1c-af05-ed82b6e68e66 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:24.262491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:24.267171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] MFA Rules not processed for user `173a41312eb940d2ab3e6b37106de975`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:24.281213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:24.348598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:24.420778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:24.421426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:24.425257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:24.426883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:24.458772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 884/1781] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:24.461874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:24.589915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a57ec-2fb3-4a4a-b84e-91cd0326a94d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:24.609933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:24.611414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:24.613269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:24.614177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:24.617261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:24.621870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:24.682378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 885/1782] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:24 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:24.687384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:24.704598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:24.712153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:24.759623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 898/1783] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:24.996140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16d39c-cdc6-4548-813f-1a374d82584e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.008844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.010652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:25.026974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.026974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.026974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.026974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.027361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.029369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.047220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ebc0f-f94c-4815-a1fb-9a9f161e5f51 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.114430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 886/1784] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:24 2022] POST /identity/v3/users => generated 296 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:25.118205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.178455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315f9e9-ad74-44b1-b2c1-c557b7ee4043 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.203269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.203902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.204174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.204438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.205088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.209775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.246619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 887/1785] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:25.250377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.282700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988306-1d83-4903-b322-9f69769fdff0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.301799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.303235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.304267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.304790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] PATH_INFO: `/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0/users/33acf18f980a4b6faec3e229384aec81/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.321532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:25.371498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f63f7cd3ec24539bb8efe2bd14533e0, user_id=33acf18f980a4b6faec3e229384aec81, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.379758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.401936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.405187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.407619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.409350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.409997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 888/1786] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:25 2022] PUT /identity/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0/users/33acf18f980a4b6faec3e229384aec81/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:25.411980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.417292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:25.606351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4def24-16d4-4d8a-a31d-d0c8cb1cd4fe admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.623007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.623680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.623958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.625143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.627264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.631296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.665501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 889/1787] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:25.670897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.684516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee45b0-a75f-4572-a768-e11a0ec5c4d5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:25.689570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:25.690685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.691128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.691877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] PATH_INFO: `/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0/users/33acf18f980a4b6faec3e229384aec81/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.694266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f63f7cd3ec24539bb8efe2bd14533e0, user_id=33acf18f980a4b6faec3e229384aec81, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.697722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106731c9-68de-4b20-a449-bd930a74b228 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.702540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:25.710127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:25.723949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 890/1788] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:25 2022] PUT /identity/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0/users/33acf18f980a4b6faec3e229384aec81/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:25.725881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc83b6-73df-4862-869f-0e56c6c041bc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:25.726412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc83b6-73df-4862-869f-0e56c6c041bc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:25.726988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc83b6-73df-4862-869f-0e56c6c041bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:25.767476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-01fc83b6-73df-4862-869f-0e56c6c041bc None None] MFA Rules not processed for user `33acf18f980a4b6faec3e229384aec81`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:25.769438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-01fc83b6-73df-4862-869f-0e56c6c041bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:25.781586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 899/1789] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:25.868362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 891/1790] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:25.963129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:25.983718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba36fcd-9644-4ab8-83b0-02615ca67eaa service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:26.057227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:26.057507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:26.057777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:26.078482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:26.115078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:26.115893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:26.116514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:26.117145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:26.164861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:26.176435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:26.201317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 900/1791] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:26.251122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:26.259464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:26.346232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:26.493338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 892/1792] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:26.525560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:26.537992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6cec-21bb-4192-a37c-1fa41fd64ad2 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:26.548123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:26.549116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:26.549403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:26.549745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:26.760882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:26.762522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:26.834410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:26.845367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517244b0-3ec6-4757-b722-28f1f19c6e9c service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:26.906008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 901/1793] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:27.049206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74775a-27c9-49d3-bae6-2eea5f586f78 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:27.061005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:27.061678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:27.062001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:27.062291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:27.073879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:27.075715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:27.087208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 893/1794] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:49:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 03:49:27.745786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:27.746493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:27.746493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:27.793426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] MFA Rules not processed for user `31a8c62bd81f4c1f863bd952b6976276`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:27.797858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:28.103225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 902/1795] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:28.140710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:28.160699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3311f4-9b50-4f6c-9702-201926ca7f06 service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:28.174733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:28.179838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:28.179838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:28.179838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:28.191854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:28.494537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:28.497571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c96b2-6267-4e32-a79e-c11dfa900e20 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:28.574991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 894/1796] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:32.527392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:32.529659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:32.664390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953134-dbf5-4d53-a9f7-8f7942272e35 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:32.677563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:32.678657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:32.679143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:32.679849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] PATH_INFO: `/v3/users/d79b846b67e94b39b3a450833c095082` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:32.681747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79b846b67e94b39b3a450833c095082)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:32.684345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:32.750900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:32.783690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79b846b67e94b39b3a450833c095082'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:32.807994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79b846b67e94b39b3a450833c095082'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:32.816899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79b846b67e94b39b3a450833c095082'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:32.817656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79b846b67e94b39b3a450833c095082'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:32.818670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79b846b67e94b39b3a450833c095082'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:32.836765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 903/1797] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:32 2022] DELETE /identity/v3/users/d79b846b67e94b39b3a450833c095082 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:32.841046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:32.841562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:32.841661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:32.863033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:32.864708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:32.919683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:32.972978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 895/1798] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:32.979523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:32.981490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:33.058499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc964b94-54ad-4eb4-89d9-de561d50115b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:33.063553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:33.064331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:33.064704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:33.065173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:33.082320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:33.083948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:33.095970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 904/1799] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:33.386949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:33.388420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:33.400622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c3f3e-fb7a-417a-bcfe-2d63441aae07 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:33.405557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:33.406387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:33.406663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:33.407103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] PATH_INFO: `/v3/projects/032a9710741b4b268a44b8b17b209c7d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:33.408416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032a9710741b4b268a44b8b17b209c7d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:33.410552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:33.487401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032a9710741b4b268a44b8b17b209c7d', 'request_id': 'req-d0dd3685-ba17-4abe-b248-d3e2878a4c03'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.501376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032a9710741b4b268a44b8b17b209c7d', 'request_id': 'req-d0dd3685-ba17-4abe-b248-d3e2878a4c03'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.511054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032a9710741b4b268a44b8b17b209c7d', 'request_id': 'req-d0dd3685-ba17-4abe-b248-d3e2878a4c03'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.524895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] The token cache is being invalidate because project 032a9710741b4b268a44b8b17b209c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:33.526851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.526851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.530547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 896/1800] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:33 2022] DELETE /identity/v3/projects/032a9710741b4b268a44b8b17b209c7d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:33.533090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:33.546145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67053e-6da2-4061-8527-5652b864dd8f service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:33.623390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:33.624087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:33.624404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:33.624674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] PATH_INFO: `/v3/users/fc20a48c71da490abe7362928812085c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:33.625929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc20a48c71da490abe7362928812085c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:33.628151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:33.694677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc20a48c71da490abe7362928812085c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.702437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:33.719351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc20a48c71da490abe7362928812085c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.733041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc20a48c71da490abe7362928812085c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.735245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc20a48c71da490abe7362928812085c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.735969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc20a48c71da490abe7362928812085c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:33.777027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 905/1801] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:33 2022] DELETE /identity/v3/users/fc20a48c71da490abe7362928812085c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:34.392248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:34.394785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:34.429143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3685-ba17-4abe-b248-d3e2878a4c03 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:34.612499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:34.613634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:34.614015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:34.619804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] PATH_INFO: `/v3/projects/ffb0ee08212d49ca98190a0a74a401e3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:34.623300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb0ee08212d49ca98190a0a74a401e3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:34.625491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:34.730263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb0ee08212d49ca98190a0a74a401e3', 'request_id': 'req-f4ba8b39-0e3b-47fe-9313-50c9787a4963'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:34.774092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb0ee08212d49ca98190a0a74a401e3', 'request_id': 'req-f4ba8b39-0e3b-47fe-9313-50c9787a4963'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:34.776210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:34.795321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb0ee08212d49ca98190a0a74a401e3', 'request_id': 'req-f4ba8b39-0e3b-47fe-9313-50c9787a4963'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:34.813752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] The token cache is being invalidate because project ffb0ee08212d49ca98190a0a74a401e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:34.814145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:34.814731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:34.819329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 897/1802] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:34 2022] DELETE /identity/v3/projects/ffb0ee08212d49ca98190a0a74a401e3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:34.820532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:34.821992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:34.845187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a6a74-d5f7-4b31-8f1c-5b8ec8e86d63 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:35.003604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:35.004973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:35.005511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:35.005984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] PATH_INFO: `/v3/users/d8d66db4c659443b8a8e374d3ab1a47c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:35.007369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d66db4c659443b8a8e374d3ab1a47c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:35.009572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:35.092332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d66db4c659443b8a8e374d3ab1a47c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.123902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d66db4c659443b8a8e374d3ab1a47c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.144490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d66db4c659443b8a8e374d3ab1a47c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.146649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d66db4c659443b8a8e374d3ab1a47c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.149732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c9bf64b1-675c-4239-a081-73539556060f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d66db4c659443b8a8e374d3ab1a47c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.190985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 906/1803] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:34 2022] DELETE /identity/v3/users/d8d66db4c659443b8a8e374d3ab1a47c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:35.624223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:35.626566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:35.675584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba8b39-0e3b-47fe-9313-50c9787a4963 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:35.751871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:35.752586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:35.752928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:35.753404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] PATH_INFO: `/v3/projects/bf78a06119f34740a3b52d17a4e37002` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:35.754618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf78a06119f34740a3b52d17a4e37002)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:35.756657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:35.805473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf78a06119f34740a3b52d17a4e37002', 'request_id': 'req-b8243dd4-4a9c-433f-b582-9f3029332b5d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.813799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf78a06119f34740a3b52d17a4e37002', 'request_id': 'req-b8243dd4-4a9c-433f-b582-9f3029332b5d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.819539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf78a06119f34740a3b52d17a4e37002', 'request_id': 'req-b8243dd4-4a9c-433f-b582-9f3029332b5d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.831408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] The token cache is being invalidate because project bf78a06119f34740a3b52d17a4e37002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:35.831779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.832445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b8243dd4-4a9c-433f-b582-9f3029332b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:35.840969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 898/1804] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:35 2022] DELETE /identity/v3/projects/bf78a06119f34740a3b52d17a4e37002 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:35.844773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:35.845338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:35.845338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:35.883037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:35.884185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:35.929843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:35.950277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 907/1805] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:35.958777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:35.959161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:35.959434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:35.979342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:35.980544 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:35.989348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 899/1806] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:35.993580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.003411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157a086-e4fe-4aec-a910-80c01f949fcf None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.008541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.009135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.009412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.009680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.010316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:36.010615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.012224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.024976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 908/1807] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:49:36.029825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.039986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e5bd-46f2-492e-b4a2-616a47617460 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.058345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.059183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.059741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.060017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.060667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.063017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.110843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 900/1808] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:36 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:36.114330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.126349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdc927-c0d9-41a0-b2d4-b036cc502553 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.184124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.184922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.185166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.185454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.186132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.188319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.220996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 909/1809] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:36 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:36.222435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.243710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e59a53-4363-4836-9343-29f898d03363 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.249616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.250127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.250420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.250730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.252225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.259080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.276168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.286994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 901/1810] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:36.295594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7528a-466c-4810-90d4-c665f8d7e35a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.302940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.303816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.304328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.304665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.305410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.307260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26e49b-402e-42da-9a95-4cfcc7e9ade7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.319731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 910/1811] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:36.324250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.337483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a40bd-0646-42d6-be24-622118636fe4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.345955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.346986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.347304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.347583 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] PATH_INFO: `/v3/projects/08981cfb4723407696b90cd3c28fe3af/users/dd2e801d808a4bac9eca2e608616c363/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.364617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08981cfb4723407696b90cd3c28fe3af, user_id=dd2e801d808a4bac9eca2e608616c363, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.367552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.407319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 902/1812] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:36 2022] PUT /identity/v3/projects/08981cfb4723407696b90cd3c28fe3af/users/dd2e801d808a4bac9eca2e608616c363/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:36.408729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84448001-ffda-4208-8a71-63fe0080565c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.409106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84448001-ffda-4208-8a71-63fe0080565c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.409426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84448001-ffda-4208-8a71-63fe0080565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.448736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-84448001-ffda-4208-8a71-63fe0080565c None None] MFA Rules not processed for user `dd2e801d808a4bac9eca2e608616c363`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:36.450077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-84448001-ffda-4208-8a71-63fe0080565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:36.535145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 911/1813] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:36.546417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:36.548149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:36.561922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:36.629527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c247-0e4a-44a4-af3d-cff48d86b66f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:36.638717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:36.639646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:36.640178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:36.640454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:36.651086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:36.652566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:36.664433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 903/1814] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:41.735987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:41.736577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:41.736577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:41.757679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:41.758923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:41.804875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:41.819323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 912/1815] 10.222.0.57 () {58 vars in 1015 bytes} [Sat Sep 3 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:41.890290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:41.890581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:41.891838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:41.892849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:41.902265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef14d-96b6-4950-bd59-53951e924610 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:41.906029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f199ee-5c11-4fd1-bf43-7d50d0a6c458 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:41.906257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:41.907381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:41.907381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:41.907568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:41.910082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:41.910952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:41.911698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:41.911827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:41.915767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:41.921821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:41.933376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:41.933936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:41.942904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 913/1816] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:41.943796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 904/1817] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:49:41.944694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:41.946943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:41.958844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a27cd-f4fb-4066-bb2a-92a407df2b47 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:41.964512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:41.965318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:41.965813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:41.966266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:41.974540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aa6-b51d-49a1-969a-113a43767321 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:41.975350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:41.976955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:41.984863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:41.985491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:41.985822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:41.986095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:41.989807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 905/1818] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:41.991672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:42.011171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:42.012003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b78ae7-8cd3-4139-8623-5e4e10f32290 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:42.012619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:42.021069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:42.024651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:42.025070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:42.026120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:42.028947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 914/1819] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:42.041424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:42.043988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d86aaa-69eb-4580-9844-6651d396bb2e service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:42.073037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 906/1820] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:43.368789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:43.370258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:43.487624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226582d1-db92-4f47-840f-19cf2ffd36b5 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:43.493503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:43.494319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:43.494970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:43.495349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] PATH_INFO: `/v3/users/173a41312eb940d2ab3e6b37106de975` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:43.496652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=173a41312eb940d2ab3e6b37106de975)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:43.499000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:43.622944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173a41312eb940d2ab3e6b37106de975'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:43.651702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173a41312eb940d2ab3e6b37106de975'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:43.700382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173a41312eb940d2ab3e6b37106de975'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:43.701429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173a41312eb940d2ab3e6b37106de975'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:43.702830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173a41312eb940d2ab3e6b37106de975'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:43.743456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e71c-4f01-4d83-b26b-93b684199893 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:43.744268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 915/1821] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:43 2022] DELETE /identity/v3/users/173a41312eb940d2ab3e6b37106de975 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:43.744659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e71c-4f01-4d83-b26b-93b684199893 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:43.744976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e71c-4f01-4d83-b26b-93b684199893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:43.778605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-da64e71c-4f01-4d83-b26b-93b684199893 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:43.783604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-da64e71c-4f01-4d83-b26b-93b684199893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:43.908581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 907/1822] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:43.966787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:43.970214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:44.062922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:44.070642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff6077-bac7-49cc-b8ae-00351af5a6de admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:44.102914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:44.104222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.104576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.104891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.116975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:44.123999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:44.133484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d72c-8601-462e-b572-0045d71a501f None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.134588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d72c-8601-462e-b572-0045d71a501f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.135230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d72c-8601-462e-b572-0045d71a501f None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.137610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 908/1823] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:49:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:49:44.139880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 916/1824] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:44.178429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a7c5-840e-46c4-ac89-b34c3ded35c0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.179072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a7c5-840e-46c4-ac89-b34c3ded35c0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.179576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7a7c5-840e-46c4-ac89-b34c3ded35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.207388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7aa7a7c5-840e-46c4-ac89-b34c3ded35c0 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:44.210473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7aa7a7c5-840e-46c4-ac89-b34c3ded35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:44.329888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 909/1825] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:49:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:44.548371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:44.557889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:44.586072 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391687-7510-4465-b583-d462e7e8d776 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:44.605976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:44.605976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.605976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.605976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.627764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:44.629721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50875a4f-219d-42f6-8ae9-b7998eb585f5 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:44.650873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 917/1826] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:44.678841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.679179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.679459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.680860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 910/1827] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:49:44.748060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:44.748240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:44.748515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:44.777607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:44.779080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:44.791343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 918/1828] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:49:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:45.492932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:45.494956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:45.513832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadd7b9-1268-41cd-8371-fe662324c437 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:45.520484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:45.521317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:45.521598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:45.521877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:45.528020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:45.532180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:45.534300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041db34-16a1-49ae-9ee2-73d52016100a service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:45.554198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 911/1829] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:45.722728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:45.725056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:45.753977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb57f3-1b25-419f-b201-b064d8874cc8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:45.763075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:45.764414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:45.764899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:45.765399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] PATH_INFO: `/v3/users/dd2e801d808a4bac9eca2e608616c363` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:45.766988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2e801d808a4bac9eca2e608616c363)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:45.770781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:45.841476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2e801d808a4bac9eca2e608616c363'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:45.858014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2e801d808a4bac9eca2e608616c363'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:45.866217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2e801d808a4bac9eca2e608616c363'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:45.872536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2e801d808a4bac9eca2e608616c363'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:45.873227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2e801d808a4bac9eca2e608616c363'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:45.894139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 919/1830] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:45 2022] DELETE /identity/v3/users/dd2e801d808a4bac9eca2e608616c363 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:45.895667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:45.896108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:45.896504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:45.917684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:45.919174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.010255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 912/1831] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:46.038328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.040308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.135676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dd8d8-d1d2-4487-b30b-f653a3b330d2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.139397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.140019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.140300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.140576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.151698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.153896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:46.169592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 920/1832] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:46.225291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.226762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.241946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199d473-c454-4288-88ca-32aecc9a8667 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.250330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.251263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.252475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.252893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] PATH_INFO: `/v3/projects/520e52fb9c0749c99fe04fcc65ee3559` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.257965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=520e52fb9c0749c99fe04fcc65ee3559)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.265480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:46.340379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520e52fb9c0749c99fe04fcc65ee3559', 'request_id': 'req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.351268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520e52fb9c0749c99fe04fcc65ee3559', 'request_id': 'req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.358933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520e52fb9c0749c99fe04fcc65ee3559', 'request_id': 'req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.374667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] The token cache is being invalidate because project 520e52fb9c0749c99fe04fcc65ee3559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:46.375307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.377810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.380352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 913/1833] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:46 2022] DELETE /identity/v3/projects/520e52fb9c0749c99fe04fcc65ee3559 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:46.383776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.394826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2279044-27ad-415d-a499-5042624e8f1a service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.443528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:46.466835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.467101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.467300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.467523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] PATH_INFO: `/v3/users/33acf18f980a4b6faec3e229384aec81` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.468668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33acf18f980a4b6faec3e229384aec81)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.471199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:46.531368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.535003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.567017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72fa7-3ae5-4e4a-98bc-dd9b6d2d9f8b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.568487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33acf18f980a4b6faec3e229384aec81'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.590757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.591887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.594047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.594538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] PATH_INFO: `/v3/projects/08981cfb4723407696b90cd3c28fe3af` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.596951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08981cfb4723407696b90cd3c28fe3af)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.600222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:46.606365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33acf18f980a4b6faec3e229384aec81'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.616520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33acf18f980a4b6faec3e229384aec81'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.617298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33acf18f980a4b6faec3e229384aec81'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.618006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-62a07c13-1c19-400b-9584-267d6142eb29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33acf18f980a4b6faec3e229384aec81'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.636292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 921/1834] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:46 2022] DELETE /identity/v3/users/33acf18f980a4b6faec3e229384aec81 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:46.674403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08981cfb4723407696b90cd3c28fe3af', 'request_id': 'req-1aba29f3-5885-4db3-8e80-a49ed780ffc3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.686652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08981cfb4723407696b90cd3c28fe3af', 'request_id': 'req-1aba29f3-5885-4db3-8e80-a49ed780ffc3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.695494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08981cfb4723407696b90cd3c28fe3af', 'request_id': 'req-1aba29f3-5885-4db3-8e80-a49ed780ffc3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.705882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:46.711113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] The token cache is being invalidate because project 08981cfb4723407696b90cd3c28fe3af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:46.711632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.713554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1aba29f3-5885-4db3-8e80-a49ed780ffc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:46.715291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 914/1835] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:46 2022] DELETE /identity/v3/projects/08981cfb4723407696b90cd3c28fe3af => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:46.721070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.721358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.721641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.744755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:46.747164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.829772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 922/1836] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:46.834983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.835310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.835744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.859077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:46.860596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:46.870957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 915/1837] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:46.875018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.891276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36d0f0-d6cb-48a3-943d-dc8c1ca2a18d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.897715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.898640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.899054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.899555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.900832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:46.901476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.904117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:46.920076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 923/1838] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:46.931520 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:46.941783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9342c-62a8-43d0-8efb-68b09d83f851 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:46.948295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:46.948918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:46.949196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:46.949472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:46.950239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:46.952212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.002920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 916/1839] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:46 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:47.003719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.020090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3f721-b8c2-4b78-8d67-93073075acfe admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.116307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:47.131574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.131574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.131860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.131860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.132750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.134955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.161212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 924/1840] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:49:47.164921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.175605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b230c-fd4a-41dc-9ab5-b3844966abc1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.180532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.181148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.181616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.181890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.182556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.184822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.200615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.201557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 917/1841] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:47.223183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837002-3df8-4009-a224-c2d7f1c6918b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.240893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.243010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.243870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.243870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.244452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.248163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.260815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.265852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:47.292312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd10f2-f759-4cf1-8717-e210e45e1dc1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.299189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 925/1842] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:47.300685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.307724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.308301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.308625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.309035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] PATH_INFO: `/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.311587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f63f7cd3ec24539bb8efe2bd14533e0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.312361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558170e-df9a-4d73-b2db-ecf773690bef admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.317248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.317858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.318111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.318380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.318662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] PATH_INFO: `/v3/projects/688533d1be1a4702ace4541926886e0d/users/02aa760da9b84854adf10d63aa89650b/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.336902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=688533d1be1a4702ace4541926886e0d, user_id=02aa760da9b84854adf10d63aa89650b, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.340411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180cc08-2268-40c1-b959-7c5f45c0de6f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.367246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 926/1843] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:47 2022] PUT /identity/v3/projects/688533d1be1a4702ace4541926886e0d/users/02aa760da9b84854adf10d63aa89650b/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:47.376445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.378493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.378890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.440158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f63f7cd3ec24539bb8efe2bd14533e0', 'request_id': 'req-eb9a4083-2d2d-4111-a884-3447b93d7e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:47.452224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f63f7cd3ec24539bb8efe2bd14533e0', 'request_id': 'req-eb9a4083-2d2d-4111-a884-3447b93d7e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:47.457834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] MFA Rules not processed for user `02aa760da9b84854adf10d63aa89650b`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:47.460061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f63f7cd3ec24539bb8efe2bd14533e0', 'request_id': 'req-eb9a4083-2d2d-4111-a884-3447b93d7e5f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:47.461484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:47.474690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] The token cache is being invalidate because project 3f63f7cd3ec24539bb8efe2bd14533e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:47.475016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:47.476350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eb9a4083-2d2d-4111-a884-3447b93d7e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:47.478232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 918/1844] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:47 2022] DELETE /identity/v3/projects/3f63f7cd3ec24539bb8efe2bd14533e0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:47.483138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e0a8a-2e41-49ae-9523-c67aa7b1260a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.483539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e0a8a-2e41-49ae-9523-c67aa7b1260a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.483913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e0a8a-2e41-49ae-9523-c67aa7b1260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.510832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8f2e0a8a-2e41-49ae-9523-c67aa7b1260a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:47.512784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8f2e0a8a-2e41-49ae-9523-c67aa7b1260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:47.578504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 927/1845] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:47.591273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.593015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:47.598358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 919/1846] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:49:47.608837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.609376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.609809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.632404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:47.634245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:47.671738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 920/1847] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:47.672691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.702460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f316e9-22eb-4b93-9ef4-cab30251044b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.717489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e75bc-3eef-4fa0-b182-ea82c9c9b46d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.727387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.728148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.730353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.731244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.731622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.732622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.732902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.733167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.734003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:47.734446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.736426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.755286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.758247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.777607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 921/1848] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:47.778591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.786198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 928/1849] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:49:47.803421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d519-b351-4192-a712-548f2134e4cd admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.815021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.816044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.816532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.820286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.832534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:47.834894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:47.885221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 922/1850] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:47.889642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:47.907523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600cb78-8d44-4665-9e16-c14376d23057 service glance] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:47.928028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:47.996504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:47.997258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:47.997611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:47.997957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:47.998786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.001442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.041556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 929/1851] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:47 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:48.045868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.052189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:48.062479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63201a2-c9e7-40a4-abf4-d4eca3257445 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.086167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.095200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.097341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.097812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.098854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.103738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.135060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 923/1852] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:48.139498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.153637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772eb6-fc8f-48dc-be5b-d1d8d604e1dd admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.163937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.164559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.164835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.165104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.165743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.167659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b0e4-c7e9-4b2a-aa5a-d4252700b3cf admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.190173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 930/1853] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:48.192284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.207070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf52a5-78c1-43c5-89a1-335887b423dc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.227037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.227793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.234949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.235250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] PATH_INFO: `/v3/projects/aa338313dc3c405b8bea1732158c54bd/users/19af543906c1454da14203026ffc9f3f/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.302252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa338313dc3c405b8bea1732158c54bd, user_id=19af543906c1454da14203026ffc9f3f, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.311334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.350606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 924/1854] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:48 2022] PUT /identity/v3/projects/aa338313dc3c405b8bea1732158c54bd/users/19af543906c1454da14203026ffc9f3f/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:48.361013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.361313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.361563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.422853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] MFA Rules not processed for user `19af543906c1454da14203026ffc9f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:48.430590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:48.582566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 931/1855] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:48.586933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.664373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0e44c-44f8-447a-9128-fbfeb84fb2cc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.669549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.670303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.670717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.671061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.671823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.674188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.705471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 925/1856] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:48 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:49:48.709648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.721123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5e6e0-c577-467d-8110-52c084c3359b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.781140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.781786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.782022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.782296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.782981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.784968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.806953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 932/1857] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:48 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:48.811204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.822555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e86f6-d34b-4ca4-8ece-b34279487b1f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.827765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.828376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.828654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.828914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.829574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.831292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.842166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 926/1858] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:48.846447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.856979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b877044-d929-4a1a-982e-6d0803ad1606 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.862151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.862936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.863222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.863499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] PATH_INFO: `/v3/projects/9195d59849704c2cb9679d6b6c2a9631/users/034e81c8e84b44c4880e935c8a51ea9a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.876170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:48.879546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9195d59849704c2cb9679d6b6c2a9631, user_id=034e81c8e84b44c4880e935c8a51ea9a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.882289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.896844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 933/1859] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:48 2022] PUT /identity/v3/projects/9195d59849704c2cb9679d6b6c2a9631/users/034e81c8e84b44c4880e935c8a51ea9a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:48.901020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:48.911781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac51f5-cf87-4eb1-b26d-5bed7ddf816e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:48.974067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:48.974929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:48.975295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:48.975691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:48.976694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:48.978420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51931564-a885-4702-94b3-2343c3b4d1f8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:48.991931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 927/1860] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:48.996222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:49.007312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f94c6-df38-48e2-8f9d-520ea6f9f9a9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:49.013882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:49.014584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:49.014883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:49.015133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] PATH_INFO: `/v3/projects/9195d59849704c2cb9679d6b6c2a9631/users/034e81c8e84b44c4880e935c8a51ea9a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:49.017262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9195d59849704c2cb9679d6b6c2a9631, user_id=034e81c8e84b44c4880e935c8a51ea9a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:49.020155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:49.035276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 934/1861] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:48 2022] PUT /identity/v3/projects/9195d59849704c2cb9679d6b6c2a9631/users/034e81c8e84b44c4880e935c8a51ea9a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:49.047881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:49.048178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:49.048452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:49.085976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] MFA Rules not processed for user `034e81c8e84b44c4880e935c8a51ea9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:49.087361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:49.116965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:49.189045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 928/1862] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:49.222983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:49.227410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:49.304507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4476-2e76-4df4-b0f3-e73c6ca2ca18 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:49.311340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:49.312070 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:49.312403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:49.312849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:49.323499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:49.325172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:49.336606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 935/1863] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:49.825707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:49.841527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5f082-712f-4f92-8db3-9d1188085127 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:49.850779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034e81c8e84b44c4880e935c8a51ea9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9195d59849704c2cb9679d6b6c2a9631', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:49.851441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:49.851717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:49.851988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:49.852646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:49.852949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:49.854797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:49.869730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 929/1864] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:49.874106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:49.885118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1a195-87f0-49af-86b7-aa2d3d051abc service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:49.891533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034e81c8e84b44c4880e935c8a51ea9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9195d59849704c2cb9679d6b6c2a9631', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:49.892427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:49.893287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:49.893794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:49.894835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:49.897422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad41a9e-7ffa-4c7a-8410-3db9015ae76a tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:49.961503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 936/1865] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:49 2022] POST /identity/v3/projects => generated 355 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:49.965614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:49.975818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b48bd-e23a-47d1-b7ed-058eb78622dd tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:50.042049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034e81c8e84b44c4880e935c8a51ea9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9195d59849704c2cb9679d6b6c2a9631', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:50.042656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.042986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.043258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.043908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:50.044172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:50.045814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7c988-8b15-4251-ad3a-68466dc21348 tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:50.058526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a1e1-f631-42e9-aa46-aa18f04c8f60 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.058742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a1e1-f631-42e9-aa46-aa18f04c8f60 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.059079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 930/1866] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:50.059193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a1e1-f631-42e9-aa46-aa18f04c8f60 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.060822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 937/1867] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:49:50.071731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.072409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.072764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.114057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532a1e1-f631-42e9-aa46-aa18f04c8f60 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:50.120218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:50.122428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:50.130691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532a1e1-f631-42e9-aa46-aa18f04c8f60 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:50.135738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034e81c8e84b44c4880e935c8a51ea9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9195d59849704c2cb9679d6b6c2a9631', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:50.136318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.136615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.137140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] PATH_INFO: `/v3/projects/d7b603de9258447fa10b9cd3ea462aef` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.138273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d7b603de9258447fa10b9cd3ea462aef)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:50.140500 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:50.222455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b603de9258447fa10b9cd3ea462aef', 'request_id': 'req-4efbf8c0-0960-4adc-88d8-45052800fe0f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.223210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 931/1868] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:50.241870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b603de9258447fa10b9cd3ea462aef', 'request_id': 'req-4efbf8c0-0960-4adc-88d8-45052800fe0f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.250691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b603de9258447fa10b9cd3ea462aef', 'request_id': 'req-4efbf8c0-0960-4adc-88d8-45052800fe0f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.270460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] The token cache is being invalidate because project d7b603de9258447fa10b9cd3ea462aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:50.271794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.272717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.285047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:50.287618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:50.292324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 938/1869] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:50 2022] DELETE /identity/v3/projects/d7b603de9258447fa10b9cd3ea462aef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:50.351657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bf1b5-f8eb-427b-a548-c391c7d0ca62 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:50.437738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:50.504815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:50.505776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.507391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.508545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.515906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:50.550661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:50.552389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f846cb-b853-467c-8fac-5cc8324e22c4 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:50.554263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:50.555295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:50.565649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 932/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:50.656134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:50.712479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbf8c0-0960-4adc-88d8-45052800fe0f tempest-VolumeQuotaClassesTest-1939263525 tempest-VolumeQuotaClassesTest-1939263525-project-admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:50.719708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:50.721355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.721883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.722352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] PATH_INFO: `/v3/users/19af543906c1454da14203026ffc9f3f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.723911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=19af543906c1454da14203026ffc9f3f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:50.726597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:50.838518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19af543906c1454da14203026ffc9f3f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.875008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19af543906c1454da14203026ffc9f3f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.885078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19af543906c1454da14203026ffc9f3f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.886034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19af543906c1454da14203026ffc9f3f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.893271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19af543906c1454da14203026ffc9f3f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:50.912725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 939/1871] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:50 2022] DELETE /identity/v3/users/19af543906c1454da14203026ffc9f3f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:50.916983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:50.917983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:50.918436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:50.950890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:50.951637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:51.045033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 933/1872] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:51.092532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:51.094622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:51.195780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040931bd-85a5-4436-9fd2-7ea01767f8ed admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:51.201711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:51.202688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:51.203246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:51.203808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:51.216286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:51.218235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:51.231919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 940/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:52.123148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:52.124931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:52.139983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc8716-f2bb-4fff-94d9-98ed549a9ac4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:52.145123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:52.145859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:52.146171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:52.146467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] PATH_INFO: `/v3/projects/aa338313dc3c405b8bea1732158c54bd` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:52.147902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa338313dc3c405b8bea1732158c54bd)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:52.150307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:52.214572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa338313dc3c405b8bea1732158c54bd', 'request_id': 'req-f536bdac-f46e-4ba4-9fe0-e04065fad953'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.225208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa338313dc3c405b8bea1732158c54bd', 'request_id': 'req-f536bdac-f46e-4ba4-9fe0-e04065fad953'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.243848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa338313dc3c405b8bea1732158c54bd', 'request_id': 'req-f536bdac-f46e-4ba4-9fe0-e04065fad953'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.266975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] The token cache is being invalidate because project aa338313dc3c405b8bea1732158c54bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:52.266975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.267285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.269449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 934/1874] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:52 2022] DELETE /identity/v3/projects/aa338313dc3c405b8bea1732158c54bd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:52.273404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:52.289622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d8ecc-f5bc-4dea-9d2c-db87590652e4 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:52.366118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:52.367050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:52.367325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:52.367568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] PATH_INFO: `/v3/users/034e81c8e84b44c4880e935c8a51ea9a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:52.368771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=034e81c8e84b44c4880e935c8a51ea9a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:52.371300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:52.376733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:52.433263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034e81c8e84b44c4880e935c8a51ea9a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.447238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034e81c8e84b44c4880e935c8a51ea9a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.455092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034e81c8e84b44c4880e935c8a51ea9a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.455832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034e81c8e84b44c4880e935c8a51ea9a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.456505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-75815e97-ecbd-44b8-bfb8-97fc49086734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034e81c8e84b44c4880e935c8a51ea9a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:52.478921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 941/1875] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:49:52 2022] DELETE /identity/v3/users/034e81c8e84b44c4880e935c8a51ea9a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:52.947940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:52.951474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:52.974677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536bdac-f46e-4ba4-9fe0-e04065fad953 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.032980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:53.054332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.054487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.054706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.055028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] PATH_INFO: `/v3/projects/9195d59849704c2cb9679d6b6c2a9631` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.056419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9195d59849704c2cb9679d6b6c2a9631)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.058639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.123509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9195d59849704c2cb9679d6b6c2a9631', 'request_id': 'req-6e18135d-f0a9-4ee0-82d1-6865680bcc55'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:53.134435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9195d59849704c2cb9679d6b6c2a9631', 'request_id': 'req-6e18135d-f0a9-4ee0-82d1-6865680bcc55'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:53.141736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9195d59849704c2cb9679d6b6c2a9631', 'request_id': 'req-6e18135d-f0a9-4ee0-82d1-6865680bcc55'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:53.162037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] The token cache is being invalidate because project 9195d59849704c2cb9679d6b6c2a9631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:49:53.162432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:53.163175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e18135d-f0a9-4ee0-82d1-6865680bcc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:49:53.168428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 935/1876] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:49:52 2022] DELETE /identity/v3/projects/9195d59849704c2cb9679d6b6c2a9631 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:53.173671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.174073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.174300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.206568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:53.207910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:53.294856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 942/1877] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:53.301338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.301792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.302212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.337217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:53.338760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:53.348564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 936/1878] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:53.352948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.371098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f89edd-4bc9-4bf5-9478-71d862f8b69e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.377696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.378383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.379044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.379327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.379999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:49:53.380307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.382008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.408441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 943/1879] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:49:53.409425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.420777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8576-12ca-46ae-90de-0a5a9dd78002 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.426592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.427521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.427962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.428241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.428900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.431458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.469199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 937/1880] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:49:53.470610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.483622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0d78d-be14-4dd7-aabc-93fb47a52615 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.566391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.567037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.567312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.567578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.568908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.570900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.599412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:53.606425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 944/1881] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:53.609774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.621940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f4669-0d78-4d0c-b572-0cf58b0134e0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.626579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.627891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.628166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.628437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.629849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.631691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.643319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 938/1882] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:49:53.647096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.659598 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc4e93-79e1-4c60-b410-c1a3edc44554 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.666956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.667591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.667871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.668327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.668987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.671866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1335d-eeec-4e7e-8dc2-22ca98fde8c4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.690432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 945/1883] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:53.691697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.704334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a180fd-6729-47e2-98b0-1682026b54e1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.709660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.710449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.710795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.711163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] PATH_INFO: `/v3/projects/79070e943dde47e7a7c9866110fbd9d2/users/1df0ca94b5284c7db5caf68865b3fc9c/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.731437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79070e943dde47e7a7c9866110fbd9d2, user_id=1df0ca94b5284c7db5caf68865b3fc9c, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.734623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.761476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 939/1884] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:53 2022] PUT /identity/v3/projects/79070e943dde47e7a7c9866110fbd9d2/users/1df0ca94b5284c7db5caf68865b3fc9c/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:53.767856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.768189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.768409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.804028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] MFA Rules not processed for user `1df0ca94b5284c7db5caf68865b3fc9c`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:53.805693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:53.890519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 946/1885] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:53.891857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:53.965857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e11fb-906b-4996-ac0a-e72f6aee6e24 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:53.970652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:53.971383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:53.971813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:53.972215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:53.972961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:53.975083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:53.991594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:54.016938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 940/1886] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:53 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:54.021018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.039462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5da36-d81c-48f7-90d2-c5d31183f85c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.150761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.151587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.151931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.152268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.152968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.154991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.178304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 947/1887] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:54 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:54.182333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.193578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03297-e306-4d05-8d43-675020240e69 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.198894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.199584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.199992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.200360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.201130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.203535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.215407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 941/1888] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:54.219661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.230291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b988-889d-4d82-b585-d492141e2f80 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.235824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.236446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.236753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.237027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.237815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.239453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05997c1a-d9cc-4dd7-a0f5-a8e2b1943d05 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.255588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 948/1889] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:54.256724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.266792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a59ba-bae3-4161-bdd6-b4a8041b1814 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.271389 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.272044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.272322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.272616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] PATH_INFO: `/v3/projects/6f14b75e2e434d58bbb13a85b37c665c/users/c1a8e6a6cddf4ed4939654c085d83bc6/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.288149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f14b75e2e434d58bbb13a85b37c665c, user_id=c1a8e6a6cddf4ed4939654c085d83bc6, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.290859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.305729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 942/1890] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:54 2022] PUT /identity/v3/projects/6f14b75e2e434d58bbb13a85b37c665c/users/c1a8e6a6cddf4ed4939654c085d83bc6/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:49:54.317146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.317618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.317743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.347320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] MFA Rules not processed for user `c1a8e6a6cddf4ed4939654c085d83bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:54.348618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:54.405757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:54.431900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 949/1891] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:54.437181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.506223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65970b1-df6d-4128-8cb3-55db4309376f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.511330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.511976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.512278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.512551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.513192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.515250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.544312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 943/1892] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:49:54 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:54.548258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.568392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d60c8b-68c2-441c-9bdb-2395fbbdee80 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.638123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.638739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.639059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.639330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.639964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.641872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.669449 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 950/1893] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:49:54 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:49:54.673550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.685378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca5e4-9dee-4096-bec6-cb85f2f32bb5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.692253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.692888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.693166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.693432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.694611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.696318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.709430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 944/1894] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:54.713961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.723924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3ac88-d9ab-4e38-85a1-e98be889708b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.728445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.729102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.729379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.729643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] PATH_INFO: `/v3/projects/b9cefd97c26947d18850bcee26e3eb81/users/26add8820f9b43afb2c5940ed6fb7292/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.746950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cefd97c26947d18850bcee26e3eb81, user_id=26add8820f9b43afb2c5940ed6fb7292, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.749653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.764365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 951/1895] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:54 2022] PUT /identity/v3/projects/b9cefd97c26947d18850bcee26e3eb81/users/26add8820f9b43afb2c5940ed6fb7292/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:54.768382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.780864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e211-e26c-4dc7-9590-a0a0c3617f72 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.840632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.841505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.841786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.842058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.842723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.844422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baecbc-6728-468e-aa9b-6ce86399c847 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.855861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 945/1896] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:49:54.860749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:54.871418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bca46-17c1-4b64-b03e-d825a517dec1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:54.876068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:54.876809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.877058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.877346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] PATH_INFO: `/v3/projects/b9cefd97c26947d18850bcee26e3eb81/users/26add8820f9b43afb2c5940ed6fb7292/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.879359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cefd97c26947d18850bcee26e3eb81, user_id=26add8820f9b43afb2c5940ed6fb7292, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:54.882048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:54.898412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 952/1897] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:49:54 2022] PUT /identity/v3/projects/b9cefd97c26947d18850bcee26e3eb81/users/26add8820f9b43afb2c5940ed6fb7292/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:49:54.909127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:54.909421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:54.909750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:54.938379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] MFA Rules not processed for user `26add8820f9b43afb2c5940ed6fb7292`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:49:54.939642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:54.959867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:55.012315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 946/1898] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:49:55.024903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:55.026783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:49:55.087464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ba6a9-1e0f-4a09-b58b-3f42d7596113 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:55.091218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:55.091854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:55.092083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:55.092388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:55.160043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:49:55.169403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:55.171980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:55.195932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 953/1899] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:57.851733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:57.861801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588965f-f812-49a9-98eb-bad216fdb071 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:57.865573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:57.866175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:57.866450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:57.866719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:57.926864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:57.929103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:57.938550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 947/1900] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:49:58.611496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:49:58.620973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647682d-4d88-490f-b4c0-fe422eff71c6 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:49:58.625314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:49:58.625927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:49:58.626208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:49:58.626479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:49:58.636493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:49:58.638046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89e993-bb11-4495-8477-d29c75af58a8 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:49:58.650168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 954/1901] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:49:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:03.901233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:03.988710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ba751-b4b1-4d9b-8dcc-5bd760dd8bf4 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:03.993768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:03.994487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:03.994865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:03.995220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] PATH_INFO: `/v3/users/1df0ca94b5284c7db5caf68865b3fc9c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:03.996419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df0ca94b5284c7db5caf68865b3fc9c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:03.998437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:04.070496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df0ca94b5284c7db5caf68865b3fc9c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.071281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df0ca94b5284c7db5caf68865b3fc9c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.083062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:04.091328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df0ca94b5284c7db5caf68865b3fc9c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.100657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df0ca94b5284c7db5caf68865b3fc9c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.101751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df0ca94b5284c7db5caf68865b3fc9c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.126146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 948/1902] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:03 2022] DELETE /identity/v3/users/1df0ca94b5284c7db5caf68865b3fc9c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:04.130255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:04.130703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:04.130793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:04.155178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:04.156984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:04.281793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 955/1903] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:04.297998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:04.299541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:04.379523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d27a-3cee-4959-bf2b-2a8ac0d85737 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:04.384349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:04.385298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:04.385889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:04.386313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:04.397568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:04.399630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:04.415434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 949/1904] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:04.764010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:04.765784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:04.781046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8d93e-4249-4282-8217-3bce39b07908 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:04.787204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:04.788058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:04.788586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:04.788989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] PATH_INFO: `/v3/projects/79070e943dde47e7a7c9866110fbd9d2` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:04.790343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79070e943dde47e7a7c9866110fbd9d2)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:04.793466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:04.863441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79070e943dde47e7a7c9866110fbd9d2', 'request_id': 'req-67b053d6-73e3-4a52-86c0-a78c641e48d2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.874520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79070e943dde47e7a7c9866110fbd9d2', 'request_id': 'req-67b053d6-73e3-4a52-86c0-a78c641e48d2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.881104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79070e943dde47e7a7c9866110fbd9d2', 'request_id': 'req-67b053d6-73e3-4a52-86c0-a78c641e48d2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.893674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] The token cache is being invalidate because project 79070e943dde47e7a7c9866110fbd9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:04.894021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.894711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:04.896586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 956/1905] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:04 2022] DELETE /identity/v3/projects/79070e943dde47e7a7c9866110fbd9d2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:04.900401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:04.912042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122c49a-383f-4404-b923-0e6d6cc32907 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:05.049749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:05.050769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:05.051243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:05.051661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] PATH_INFO: `/v3/users/c1a8e6a6cddf4ed4939654c085d83bc6` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:05.053266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a8e6a6cddf4ed4939654c085d83bc6)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:05.055574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:05.127406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a8e6a6cddf4ed4939654c085d83bc6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:05.128273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a8e6a6cddf4ed4939654c085d83bc6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:05.145626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a8e6a6cddf4ed4939654c085d83bc6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:05.153856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a8e6a6cddf4ed4939654c085d83bc6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:05.154970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a8e6a6cddf4ed4939654c085d83bc6'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:05.159703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:05.187324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 950/1906] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:04 2022] DELETE /identity/v3/users/c1a8e6a6cddf4ed4939654c085d83bc6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:06.412269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:06.414495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:06.430679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b053d6-73e3-4a52-86c0-a78c641e48d2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:06.525288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:06.526400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:06.526914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:06.527418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] PATH_INFO: `/v3/projects/6f14b75e2e434d58bbb13a85b37c665c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:06.528819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f14b75e2e434d58bbb13a85b37c665c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:06.530973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:06.539813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:06.610194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f14b75e2e434d58bbb13a85b37c665c', 'request_id': 'req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.625597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f14b75e2e434d58bbb13a85b37c665c', 'request_id': 'req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.631948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f14b75e2e434d58bbb13a85b37c665c', 'request_id': 'req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.644004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] The token cache is being invalidate because project 6f14b75e2e434d58bbb13a85b37c665c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:06.644328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.645091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.646966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 957/1907] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:06 2022] DELETE /identity/v3/projects/6f14b75e2e434d58bbb13a85b37c665c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:06.651092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:06.652967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:06.666664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7c1f-5e10-4b05-b6bf-bf4fe887a2c4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:06.730718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:06.731580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:06.732667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:06.732931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] PATH_INFO: `/v3/users/26add8820f9b43afb2c5940ed6fb7292` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:06.734890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26add8820f9b43afb2c5940ed6fb7292)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:06.738031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:06.843571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26add8820f9b43afb2c5940ed6fb7292'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.844318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26add8820f9b43afb2c5940ed6fb7292'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.859145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26add8820f9b43afb2c5940ed6fb7292'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.867126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26add8820f9b43afb2c5940ed6fb7292'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.867946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-876ea075-953a-4adf-a8bd-f1102ff14808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26add8820f9b43afb2c5940ed6fb7292'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:06.886362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 951/1908] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:06 2022] DELETE /identity/v3/users/26add8820f9b43afb2c5940ed6fb7292 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:07.523725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:07.526076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:07.539580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98cd7d-234e-4dfd-aaa6-0108efcccb76 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:07.618752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:07.620401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:07.620779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:07.621222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] PATH_INFO: `/v3/projects/b9cefd97c26947d18850bcee26e3eb81` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:07.625103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cefd97c26947d18850bcee26e3eb81)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:07.629722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:07.725886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cefd97c26947d18850bcee26e3eb81', 'request_id': 'req-3e8dd168-848f-4824-b07d-83f038176f48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:07.736262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cefd97c26947d18850bcee26e3eb81', 'request_id': 'req-3e8dd168-848f-4824-b07d-83f038176f48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:07.743905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cefd97c26947d18850bcee26e3eb81', 'request_id': 'req-3e8dd168-848f-4824-b07d-83f038176f48'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:07.757719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] The token cache is being invalidate because project b9cefd97c26947d18850bcee26e3eb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:07.758016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:07.758693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-3e8dd168-848f-4824-b07d-83f038176f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:07.760608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 958/1909] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:07 2022] DELETE /identity/v3/projects/b9cefd97c26947d18850bcee26e3eb81 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:07.765558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:07.766056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:07.766264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:07.792770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:07.793997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:07.863133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 952/1910] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:07.871106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:07.871464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:07.871897 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:07.900059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:07.901384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:07.909561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 959/1911] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:07.914037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:07.926362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503209f-6ea2-4011-9b61-6bc6c447ec44 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:07.931571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:07.932289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:07.932628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:07.932963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:07.933980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:07.934338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:07.936065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:07.950178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 953/1912] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:07.957310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:07.970834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028585c-54aa-4b95-af76-d6e04741306e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:07.975762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:07.976369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:07.976643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:07.976911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:07.977544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:07.979496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:07.997507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:08.015542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 960/1913] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:07 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:08.019458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:08.024980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:08.031162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff493354-e44f-4793-a5da-7fd974c3526d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:08.100000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:08.100662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.100912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.101178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.101816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:08.103851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:08.130198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 954/1914] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:08 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:08.134334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:08.145626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1740e8-4283-4e87-a0a4-94f5f4a32721 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:08.150487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:08.151250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.151531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.151874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.152443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:08.154059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:08.166479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 961/1915] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:08.170268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:08.180956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af02ae-db3b-4f1c-accc-8b77818cf260 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:08.185860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:08.186559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.186933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.187271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.187989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:08.189681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd08b58-3ad7-49e9-a60c-6b4a877c7cd9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:08.201825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 955/1916] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:08.206097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:08.216734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1255247-1f72-4730-a7fa-c2bbb68e5888 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:08.221499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:08.222217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.222497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.222772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] PATH_INFO: `/v3/projects/8dc12b2499eb4f8ab6f3e216b3f6ff3e/users/658ef1a43f7d445a920e5c4f2b26e339/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.240088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc12b2499eb4f8ab6f3e216b3f6ff3e, user_id=658ef1a43f7d445a920e5c4f2b26e339, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:08.242928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:08.259652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 962/1917] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:08 2022] PUT /identity/v3/projects/8dc12b2499eb4f8ab6f3e216b3f6ff3e/users/658ef1a43f7d445a920e5c4f2b26e339/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:08.264955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340982f8-a84c-490f-a3fc-9e7a8a300d98 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.265260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340982f8-a84c-490f-a3fc-9e7a8a300d98 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.265561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340982f8-a84c-490f-a3fc-9e7a8a300d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.285245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-340982f8-a84c-490f-a3fc-9e7a8a300d98 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:08.286477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-340982f8-a84c-490f-a3fc-9e7a8a300d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:08.348140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 956/1918] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:08.358765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:08.361085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:08.423329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c97b-e066-46b9-8784-b41afc6c8127 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:08.427590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:08.428211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:08.428488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:08.428760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:08.438049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:08.439509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:08.449470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 963/1919] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:09.203245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44b85-ce95-498b-80ce-c16b7bf2f163 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:09.203745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44b85-ce95-498b-80ce-c16b7bf2f163 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:09.204093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44b85-ce95-498b-80ce-c16b7bf2f163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:09.234432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f6d44b85-ce95-498b-80ce-c16b7bf2f163 None None] MFA Rules not processed for user `658ef1a43f7d445a920e5c4f2b26e339`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:09.235730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f6d44b85-ce95-498b-80ce-c16b7bf2f163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:09.311241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 957/1920] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:09.325932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:09.328027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:09.397233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb754-113f-462b-8f38-8ab434dc3932 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:09.404213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:09.404862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:09.405138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:09.405403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:09.414488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:09.415958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:09.426226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 964/1921] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:15.074947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31dc1c-eee5-4d6d-ba19-97b4c4209d21 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:15.075450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31dc1c-eee5-4d6d-ba19-97b4c4209d21 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:15.075557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31dc1c-eee5-4d6d-ba19-97b4c4209d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:15.095344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ab31dc1c-eee5-4d6d-ba19-97b4c4209d21 None None] MFA Rules not processed for user `658ef1a43f7d445a920e5c4f2b26e339`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:15.096629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ab31dc1c-eee5-4d6d-ba19-97b4c4209d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:15.104083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 958/1922] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:16.915502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:16.916950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:16.931258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addb3d-1a59-4369-b206-b771a5111eef service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:16.936171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:16.936905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:16.937245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:16.937574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] PATH_INFO: `/v3/users/a3644258235e4eb9840a74014a8e803a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:16.938974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3644258235e4eb9840a74014a8e803a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:16.940999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:16.989659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3644258235e4eb9840a74014a8e803a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:16.999959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:17.002274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3644258235e4eb9840a74014a8e803a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.010599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3644258235e4eb9840a74014a8e803a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.011823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3644258235e4eb9840a74014a8e803a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.012540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3644258235e4eb9840a74014a8e803a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.026576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 965/1923] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:16 2022] DELETE /identity/v3/users/a3644258235e4eb9840a74014a8e803a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:17.031189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:17.031679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:17.031957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:17.040971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:17.053781 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:17.055029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:17.114560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 959/1924] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:17.125096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:17.126744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:17.185149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd528-9967-47ad-acca-fccde66ccbb6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:17.189306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:17.190161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:17.190513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:17.190869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:17.201012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:17.202522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:17.214225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 966/1925] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:17.552690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:17.554099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:17.567997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf32486-c348-4275-8ed5-a2e4dee999de None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:17.573599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:17.574130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:17.575311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:17.575736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] PATH_INFO: `/v3/projects/59cdca9d9a0c4036ad29202a90a35e8e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:17.577322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59cdca9d9a0c4036ad29202a90a35e8e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:17.579266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:17.632818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59cdca9d9a0c4036ad29202a90a35e8e', 'request_id': 'req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.645843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59cdca9d9a0c4036ad29202a90a35e8e', 'request_id': 'req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.654370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59cdca9d9a0c4036ad29202a90a35e8e', 'request_id': 'req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.665901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] The token cache is being invalidate because project 59cdca9d9a0c4036ad29202a90a35e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:17.666191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.666962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.671425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 960/1926] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:17 2022] DELETE /identity/v3/projects/59cdca9d9a0c4036ad29202a90a35e8e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:17.672982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:17.686304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8f1d-66de-4fa5-9edd-114f6f5f4bcc service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:17.755892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:17.756633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:17.756983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:17.757861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] PATH_INFO: `/v3/users/3d2748de3dcd41ddaed16a6b1937934a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:17.758486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2748de3dcd41ddaed16a6b1937934a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:17.760519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:17.819609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2748de3dcd41ddaed16a6b1937934a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.837090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2748de3dcd41ddaed16a6b1937934a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.845832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2748de3dcd41ddaed16a6b1937934a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.846637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2748de3dcd41ddaed16a6b1937934a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.847262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4216351b-edba-4aac-8727-807320e2eaf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2748de3dcd41ddaed16a6b1937934a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:17.862539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 967/1927] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:17 2022] DELETE /identity/v3/users/3d2748de3dcd41ddaed16a6b1937934a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:17.864932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:18.443831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:18.445425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:18.458977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0208e-d52e-40d7-b7bd-133f1bb667c9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:18.531089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:18.531841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.532154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.532468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] PATH_INFO: `/v3/projects/d4c7a26c654f48a6ae03590c861df9e2` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.533902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c7a26c654f48a6ae03590c861df9e2)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:18.535988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:18.574026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:18.588848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c7a26c654f48a6ae03590c861df9e2', 'request_id': 'req-022aed63-b88c-48d4-a605-310538b67e66'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:18.597362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c7a26c654f48a6ae03590c861df9e2', 'request_id': 'req-022aed63-b88c-48d4-a605-310538b67e66'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:18.603294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c7a26c654f48a6ae03590c861df9e2', 'request_id': 'req-022aed63-b88c-48d4-a605-310538b67e66'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:18.615031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] The token cache is being invalidate because project d4c7a26c654f48a6ae03590c861df9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:18.615322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:18.615966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-022aed63-b88c-48d4-a605-310538b67e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:18.617606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 961/1928] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:18 2022] DELETE /identity/v3/projects/d4c7a26c654f48a6ae03590c861df9e2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:18.622711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.623212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.623302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.644135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:18.645338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:18.705538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 968/1929] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:18.714848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.715215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.715491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.737083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:18.738221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:18.748243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 962/1930] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:18.755087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:18.765630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613bb38-a835-4c80-b7a4-d62005de8aae None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:18.771195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:18.771829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.772108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.772374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.773017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:18.773315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:18.774944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:18.787197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 969/1931] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:18.793859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:18.808293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a866-3edd-4966-a6c2-4b2008abde98 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:18.816133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:18.816738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.817016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.817285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.817922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:18.819884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:18.849198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 963/1932] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:18 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:18.855390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:18.868567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f5029-0ab2-47ca-ae32-feda9fcfb3fb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:18.929077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:18.929686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.929980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.930288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.931000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:18.932962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:18.955144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 970/1933] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:18 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:18.959446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:18.970025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d123-fcf3-41f5-8082-1038f0eef174 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:18.974939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:18.975532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:18.975803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:18.976074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:18.976712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:18.978316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:18.988755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 964/1934] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:18.992609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.002917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebfe88-398b-4b35-9f6a-633b6f548359 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.007816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.008412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.008690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.008960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.009595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.011227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe8250-6952-49b8-ad8f-2a99a0738e72 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.021374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 971/1935] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:19.025830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.036094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0d7f9-cc78-4e0a-a923-126fff6813a4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.041163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.041898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.042184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.042645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] PATH_INFO: `/v3/projects/2344409bc84d43ef9f5517f2382d828f/users/404e76d197f84f4fa6070fb81c211302/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.058027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2344409bc84d43ef9f5517f2382d828f, user_id=404e76d197f84f4fa6070fb81c211302, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.060721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.077430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 965/1936] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:19 2022] PUT /identity/v3/projects/2344409bc84d43ef9f5517f2382d828f/users/404e76d197f84f4fa6070fb81c211302/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:19.088198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.088481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.088750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.127123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] MFA Rules not processed for user `404e76d197f84f4fa6070fb81c211302`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:19.128338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:19.183244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:19.193255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 972/1937] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:19.197578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.259252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78c800-da94-45b6-8a79-ba5afbc0abae admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.264022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.264650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.264901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.265168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.265803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.267746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.295666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 966/1938] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:19 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:19.300193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.311587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07261bee-8931-4da7-8a6d-1745f8839d12 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.368912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.369510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.369783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.370051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.370673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.372602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.395930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 973/1939] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:19 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:19.400029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.411360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23c09-6fe7-4e29-b14a-c9a9d1bb41c1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.416620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.417230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.417504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.417774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.418417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.420068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.431057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 967/1940] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:19.435502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.445903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5ea53-6edd-45c1-a075-c42d8bd696b9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.451051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.451718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.451993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.452265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] PATH_INFO: `/v3/projects/8c07abe51bec403791ca909871445e3b/users/41859003cd9544ef8490e3592efddc39/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.467836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c07abe51bec403791ca909871445e3b, user_id=41859003cd9544ef8490e3592efddc39, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.470538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.485256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 974/1941] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:19 2022] PUT /identity/v3/projects/8c07abe51bec403791ca909871445e3b/users/41859003cd9544ef8490e3592efddc39/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:19.489402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.499856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896dc6-a2b4-4cd4-810a-cd55fd153ccb admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.526101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:19.564283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.564980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.565277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.565532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.566175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.567806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a14fb5-66b5-41d2-886e-f255103447fc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.577725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 968/1942] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:19.582018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.592737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c1277-edcf-4af6-b323-f2a3405fbcff admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.598005 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.599384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.599677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.599951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] PATH_INFO: `/v3/projects/8c07abe51bec403791ca909871445e3b/users/41859003cd9544ef8490e3592efddc39/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.602103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c07abe51bec403791ca909871445e3b, user_id=41859003cd9544ef8490e3592efddc39, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.604855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.616871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 975/1943] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:19 2022] PUT /identity/v3/projects/8c07abe51bec403791ca909871445e3b/users/41859003cd9544ef8490e3592efddc39/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:19.628223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.628538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.628857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.661508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] MFA Rules not processed for user `41859003cd9544ef8490e3592efddc39`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:19.662736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:19.731001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 969/1944] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:19.742925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:19.744999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:19.809106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffff70d-fd0a-41ab-8103-27aa652f6bac admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:19.813113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:19.813694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:19.813968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:19.814237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:19.823861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:19.825280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6175bdb-e6a6-48cf-a205-cb468e8e337a service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:19.835033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 976/1945] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:20.121343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:20.183656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670191d-24d0-4265-852f-ee6e30bc7812 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:20.188451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:20.189075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:20.189353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:20.189612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] PATH_INFO: `/v3/users/404e76d197f84f4fa6070fb81c211302` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:20.190772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404e76d197f84f4fa6070fb81c211302)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:20.192720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:20.239372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404e76d197f84f4fa6070fb81c211302'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.240026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404e76d197f84f4fa6070fb81c211302'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.244512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:20.260984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404e76d197f84f4fa6070fb81c211302'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.267779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404e76d197f84f4fa6070fb81c211302'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.268582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404e76d197f84f4fa6070fb81c211302'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.283450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 970/1946] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:20 2022] DELETE /identity/v3/users/404e76d197f84f4fa6070fb81c211302 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:20.287946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:20.288602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:20.288602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:20.307755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:20.308953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:20.360374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:20.376644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 977/1947] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:20.387147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:20.388627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:20.451282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69547cad-2fad-484f-b220-de92f8a47c12 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:20.454864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:20.455527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:20.455828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:20.456091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:20.465066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:20.466493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:20.476412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 971/1948] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:20.757183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:20.759496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:20.772935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255e47a-0adf-49c0-9e4a-89649423bc7a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:20.779685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:20.780686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:20.781166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:20.781637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] PATH_INFO: `/v3/projects/2344409bc84d43ef9f5517f2382d828f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:20.783136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2344409bc84d43ef9f5517f2382d828f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:20.785385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:20.837472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2344409bc84d43ef9f5517f2382d828f', 'request_id': 'req-1d03c51b-2e68-4b6d-96be-5eed03704e20'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.849998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2344409bc84d43ef9f5517f2382d828f', 'request_id': 'req-1d03c51b-2e68-4b6d-96be-5eed03704e20'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.857711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2344409bc84d43ef9f5517f2382d828f', 'request_id': 'req-1d03c51b-2e68-4b6d-96be-5eed03704e20'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.870707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] The token cache is being invalidate because project 2344409bc84d43ef9f5517f2382d828f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:20.870927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.871664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:20.873158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 978/1949] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:20 2022] DELETE /identity/v3/projects/2344409bc84d43ef9f5517f2382d828f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:20.877439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:20.887510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486685a-2074-436f-95f0-b631c9e1b27c service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:20.945468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:20.946152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:20.946412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:20.946683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] PATH_INFO: `/v3/users/41859003cd9544ef8490e3592efddc39` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:20.947869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41859003cd9544ef8490e3592efddc39)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:20.949880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.000537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41859003cd9544ef8490e3592efddc39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.001185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41859003cd9544ef8490e3592efddc39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.013747 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41859003cd9544ef8490e3592efddc39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.022051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41859003cd9544ef8490e3592efddc39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.022974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fc7db42c-92fd-42f6-9894-09c98bb04927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41859003cd9544ef8490e3592efddc39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.025076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:21.043304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 972/1950] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:20 2022] DELETE /identity/v3/users/41859003cd9544ef8490e3592efddc39 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:21.323496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.325040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:21.339210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c51b-2e68-4b6d-96be-5eed03704e20 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.398443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.399050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.399442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.399732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] PATH_INFO: `/v3/projects/8c07abe51bec403791ca909871445e3b` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.400849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c07abe51bec403791ca909871445e3b)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.402889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.456351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c07abe51bec403791ca909871445e3b', 'request_id': 'req-f72b8923-e911-4890-873a-30d4c513765f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.467853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c07abe51bec403791ca909871445e3b', 'request_id': 'req-f72b8923-e911-4890-873a-30d4c513765f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.479509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c07abe51bec403791ca909871445e3b', 'request_id': 'req-f72b8923-e911-4890-873a-30d4c513765f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.493814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] The token cache is being invalidate because project 8c07abe51bec403791ca909871445e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:21.494100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.494880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f72b8923-e911-4890-873a-30d4c513765f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:21.496393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 979/1951] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:21 2022] DELETE /identity/v3/projects/8c07abe51bec403791ca909871445e3b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:21.501366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.501741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.502030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.521755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:21.522999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:21.581854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 973/1952] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:21.589588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.589872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.590147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.599196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:21.613000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:21.614227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:21.623478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 980/1953] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:21.627568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.637542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73906-1d55-4f21-adc9-33961ed51f0a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.642601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.643233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.643504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.643767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.644438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:21.644733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.646341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.657809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 974/1954] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:21.662221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.671619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094d88-da56-448c-a181-f7f81d0a5e46 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.676444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.677040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.677313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.677583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.678206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.680116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.709505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 981/1955] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:21 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:21.713573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.723702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355767d-dbcf-4f46-8668-2720b663856f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.780992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.781592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.781867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.782132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.782755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.784811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.810005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 975/1956] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:21 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:21.815040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.828242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b616fb-ba99-43be-95ea-579c4b3054ae admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.833394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.834007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.834282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.834555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.835358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.836939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.847920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 982/1957] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:21.851993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.862430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993226d0-b148-40e9-82b0-11efd68fbe9c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.867070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.867669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.867943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.868209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.868844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.870426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c3f99-6874-46b5-8584-879f15890e65 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.881844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 976/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:21.886061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:21.896898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea0a85-0059-458b-8962-990f96c96c61 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:21.902125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:21.902785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.903092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.903363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] PATH_INFO: `/v3/projects/2f8b3d783b6849ba972cbac8b206574c/users/99d95cedc5084fa2ac7aad87d5724c04/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.919668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8b3d783b6849ba972cbac8b206574c, user_id=99d95cedc5084fa2ac7aad87d5724c04, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:21.922234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:21.936712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 983/1959] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:21 2022] PUT /identity/v3/projects/2f8b3d783b6849ba972cbac8b206574c/users/99d95cedc5084fa2ac7aad87d5724c04/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:21.947284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:21.947730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:21.948647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:21.976173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] MFA Rules not processed for user `99d95cedc5084fa2ac7aad87d5724c04`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:21.977415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:22.022292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:22.042870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 977/1960] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:22.047359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.110173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390722b-6054-4536-b866-105f86355b45 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.115927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.116701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.116981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.117251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.117887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.119843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.153884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 984/1961] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:22.158153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.170387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ca3c-be99-40f5-a87e-53f15ddd36f7 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.230223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.230828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.231220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.231487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.232437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.234365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.265630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 978/1962] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:22.266848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.277924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597174-ecef-44b5-9e5b-aabce2ff28d0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.283675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.284413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.284752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.285025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.285638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.287337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.298229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 985/1963] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:22.302171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.312296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685909ea-87a5-489d-b724-c5293ac4967d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.317303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.317932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.318230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.318496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.319170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.320773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771c4d4-0ed3-4458-92bc-1835315677e8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.334454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 979/1964] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:22.335355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.348442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae6e02-dffe-41c2-b265-a6d224f7c5a4 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.353336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.354018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.354285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.354586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] PATH_INFO: `/v3/projects/1beb0d6c8d1845e3b7f9ce06eb3694a0/users/02bba3661d074fa9873279325484c3c5/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.369891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1beb0d6c8d1845e3b7f9ce06eb3694a0, user_id=02bba3661d074fa9873279325484c3c5, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.372632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.387067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 986/1965] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:22 2022] PUT /identity/v3/projects/1beb0d6c8d1845e3b7f9ce06eb3694a0/users/02bba3661d074fa9873279325484c3c5/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:22.398433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.398784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.399137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.428942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] MFA Rules not processed for user `02bba3661d074fa9873279325484c3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:22.430320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:22.495260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 980/1966] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:22.499780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.555680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:22.568980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bd6d-e3b8-4b23-8885-0bfe8231fd66 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.574514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.575176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.575506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.575786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.576451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.578457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.605677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 987/1967] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:22.612865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.624465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929158b-329a-45cc-86ef-c87da27b6655 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.680006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:22.688632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.689322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.689605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.689915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.690550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.692549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.716502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 981/1968] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:22.720362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.730909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e08c4-ab95-420c-8e0f-24b0adb347ff admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.735814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.736442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.736799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.737059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.737702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.739378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.751367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 988/1969] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:22.755558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.765485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675769-a56a-41d0-847b-1b3b676a42d4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.770138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.771562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.771562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.771562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] PATH_INFO: `/v3/projects/1bec2f81292446bfa3246b679ea82e7c/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.786505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bec2f81292446bfa3246b679ea82e7c, user_id=bb7e2fd5b3eb417ea7fdd2ff369e3e3e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.789230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.804216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 982/1970] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:22 2022] PUT /identity/v3/projects/1bec2f81292446bfa3246b679ea82e7c/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:22.808197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.819495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd669f0-26f5-423a-9223-8b6869b453ab admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.878019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.878639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.878955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.879222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.879855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.881454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af456e7f-c013-42fa-99c6-3c4b17756984 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.891089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 989/1971] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:22.895420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:22.905842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87f8ea-29a9-4830-9db2-4519e4651880 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:22.911304 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:22.911947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.912215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.912476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] PATH_INFO: `/v3/projects/1bec2f81292446bfa3246b679ea82e7c/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.914423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bec2f81292446bfa3246b679ea82e7c, user_id=bb7e2fd5b3eb417ea7fdd2ff369e3e3e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:22.917081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:22.927557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 983/1972] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:22 2022] PUT /identity/v3/projects/1bec2f81292446bfa3246b679ea82e7c/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:22.938404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:22.938711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:22.939022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:22.969915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] MFA Rules not processed for user `bb7e2fd5b3eb417ea7fdd2ff369e3e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:22.971260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:23.058952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 990/1973] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:23.073604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:23.075433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:23.142643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3538e14-2b13-44e8-9252-d92e2a8553a1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:23.147250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:23.148070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:23.148543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:23.149066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:23.160334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:23.161806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:23.173966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 984/1974] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:23.217763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:23.219320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:23.231768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a62ff-6dd7-471f-8a6f-765180f5cdba None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:23.235675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:23.236282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:23.236578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:23.236872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:23.281547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:23.296693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:23.298180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4ef6b-9f2f-451a-82fe-7f4810515501 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:23.307253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 991/1975] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:26.322089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:26.405049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfa570-f999-4ba1-9029-97d2b103aad8 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:26.410055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:26.410708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:26.411097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:26.411370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] PATH_INFO: `/v3/users/99d95cedc5084fa2ac7aad87d5724c04` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:26.412494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d95cedc5084fa2ac7aad87d5724c04)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:26.414442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:26.462545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d95cedc5084fa2ac7aad87d5724c04'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:26.463244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d95cedc5084fa2ac7aad87d5724c04'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:26.467756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:26.477044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d95cedc5084fa2ac7aad87d5724c04'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:26.484305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d95cedc5084fa2ac7aad87d5724c04'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:26.484987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d95cedc5084fa2ac7aad87d5724c04'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:26.498229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 985/1976] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:26 2022] DELETE /identity/v3/users/99d95cedc5084fa2ac7aad87d5724c04 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:26.502702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:26.503207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:26.503296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:26.521752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:26.522975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:26.583855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 992/1977] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:26.593423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:26.594961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:26.664057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f50c-0e3e-45be-9664-ee2feb5af441 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:26.667914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:26.668676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:26.668957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:26.669257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:26.678772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:26.680155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:26.689710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 986/1978] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:26.974186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:26.975704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:26.988199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf1a4-0cca-4018-b044-e635e133ba56 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:26.993456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:26.994097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:26.994375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:26.994641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] PATH_INFO: `/v3/projects/2f8b3d783b6849ba972cbac8b206574c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:26.995809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8b3d783b6849ba972cbac8b206574c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:26.997694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.046609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8b3d783b6849ba972cbac8b206574c', 'request_id': 'req-0c67e533-cb1c-4330-851b-c6d33d90b333'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.054937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.055476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8b3d783b6849ba972cbac8b206574c', 'request_id': 'req-0c67e533-cb1c-4330-851b-c6d33d90b333'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.056432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.061437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8b3d783b6849ba972cbac8b206574c', 'request_id': 'req-0c67e533-cb1c-4330-851b-c6d33d90b333'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.069408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7477-14bb-42fd-86b7-afc2411f6cd3 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.073549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] The token cache is being invalidate because project 2f8b3d783b6849ba972cbac8b206574c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:27.073838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.074840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.076229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 993/1979] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:26 2022] DELETE /identity/v3/projects/2f8b3d783b6849ba972cbac8b206574c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:27.079775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.081215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.128108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.128670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.128949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.129227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] PATH_INFO: `/v3/users/02aa760da9b84854adf10d63aa89650b` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.130936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] RBAC: Authorizing `identity:delete_user(user_id=02aa760da9b84854adf10d63aa89650b)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.132947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.148568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67e533-cb1c-4330-851b-c6d33d90b333 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.153525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.154165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.154440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.154711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] PATH_INFO: `/v3/users/02bba3661d074fa9873279325484c3c5` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.155858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bba3661d074fa9873279325484c3c5)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.157823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.169424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:27.189973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02aa760da9b84854adf10d63aa89650b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.190942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02aa760da9b84854adf10d63aa89650b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.208589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02aa760da9b84854adf10d63aa89650b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.209144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bba3661d074fa9873279325484c3c5'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.217067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02aa760da9b84854adf10d63aa89650b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.217849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54ebc537-534c-415f-bb54-4ab950c7700c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02aa760da9b84854adf10d63aa89650b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.219815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:27.223900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bba3661d074fa9873279325484c3c5'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.234238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bba3661d074fa9873279325484c3c5'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.236097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 987/1980] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:27 2022] DELETE /identity/v3/users/02aa760da9b84854adf10d63aa89650b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:27.239770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bba3661d074fa9873279325484c3c5'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.240525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bba3661d074fa9873279325484c3c5'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.242356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.242891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.243370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.264848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 994/1981] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:27 2022] DELETE /identity/v3/users/02bba3661d074fa9873279325484c3c5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:27.267661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:27.269280 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.331610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 988/1982] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:27.342446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.343946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.404629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd4b0e-ec21-4b35-a8a1-447447ea9843 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.412297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.412942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.413225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.413499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.425155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.426602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.440725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 995/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:27.631291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.632997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.644609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06668ac7-bc7f-4bce-a4dc-a167d13b1a14 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.649248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.649830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.650049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.650322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] PATH_INFO: `/v3/projects/1beb0d6c8d1845e3b7f9ce06eb3694a0` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.655593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1beb0d6c8d1845e3b7f9ce06eb3694a0)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.657612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.715333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1beb0d6c8d1845e3b7f9ce06eb3694a0', 'request_id': 'req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.725489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1beb0d6c8d1845e3b7f9ce06eb3694a0', 'request_id': 'req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.734281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1beb0d6c8d1845e3b7f9ce06eb3694a0', 'request_id': 'req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.753936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] The token cache is being invalidate because project 1beb0d6c8d1845e3b7f9ce06eb3694a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:27.754945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.755938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.758503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 989/1984] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:27 2022] DELETE /identity/v3/projects/1beb0d6c8d1845e3b7f9ce06eb3694a0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:27.762571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.772699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6405-c5b8-4c83-98cb-62f3053f4bd3 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.839045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.839697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.839980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.840261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] PATH_INFO: `/v3/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.841400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7e2fd5b3eb417ea7fdd2ff369e3e3e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.844484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.901832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:27.921778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7e2fd5b3eb417ea7fdd2ff369e3e3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.942219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:27.943233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7e2fd5b3eb417ea7fdd2ff369e3e3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.943668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:27.958121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808c505-591b-48c4-b88b-d4f6fb3ccd1e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:27.963076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7e2fd5b3eb417ea7fdd2ff369e3e3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.964278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7e2fd5b3eb417ea7fdd2ff369e3e3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.965318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e99918b2-f09c-425b-866b-281d1d234414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7e2fd5b3eb417ea7fdd2ff369e3e3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:27.968596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:27.969302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:27.969579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:27.970442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] PATH_INFO: `/v3/projects/688533d1be1a4702ace4541926886e0d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:27.972224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688533d1be1a4702ace4541926886e0d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:27.974241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:27.997165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 996/1985] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:27 2022] DELETE /identity/v3/users/bb7e2fd5b3eb417ea7fdd2ff369e3e3e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:28.050994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688533d1be1a4702ace4541926886e0d', 'request_id': 'req-3919c18d-7b24-4b4a-8ec7-e44de2d36558'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.061704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688533d1be1a4702ace4541926886e0d', 'request_id': 'req-3919c18d-7b24-4b4a-8ec7-e44de2d36558'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.070232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688533d1be1a4702ace4541926886e0d', 'request_id': 'req-3919c18d-7b24-4b4a-8ec7-e44de2d36558'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.081265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=3,keystone:DELETE=15,keystone:SELECT=82 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:28.086076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] The token cache is being invalidate because project 688533d1be1a4702ace4541926886e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:28.086713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.087814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3919c18d-7b24-4b4a-8ec7-e44de2d36558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.092170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 990/1986] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:27 2022] DELETE /identity/v3/projects/688533d1be1a4702ace4541926886e0d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:28.095904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.096196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.096477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.118782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:28.120081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.218842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 997/1987] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:28.226631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.226953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.227227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.271980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:28.273241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.285338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 991/1988] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:28.287940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.302571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edcb62-fb86-4095-b51b-af6e8a9c5705 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.323299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.324082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.324381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.324801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.325483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:28.325785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.327931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.335811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.338034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.346705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 998/1989] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:28.348128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.361024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d790ad-fb86-4737-9c05-62847ad62f03 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.361293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246e174-1404-4dc2-9f07-2938db5694c1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.366478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.367355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.367683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.367975 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.368777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.371384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.377269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.378569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.379054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.379464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] PATH_INFO: `/v3/projects/1bec2f81292446bfa3246b679ea82e7c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.380953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bec2f81292446bfa3246b679ea82e7c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.384182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.410068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 999/1990] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:28.412794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.451329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bec2f81292446bfa3246b679ea82e7c', 'request_id': 'req-19df6810-59c3-4218-ab0c-e11e69d3f177'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.463721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bec2f81292446bfa3246b679ea82e7c', 'request_id': 'req-19df6810-59c3-4218-ab0c-e11e69d3f177'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.487961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bec2f81292446bfa3246b679ea82e7c', 'request_id': 'req-19df6810-59c3-4218-ab0c-e11e69d3f177'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.504250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] The token cache is being invalidate because project 1bec2f81292446bfa3246b679ea82e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:28.504590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.505229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-19df6810-59c3-4218-ab0c-e11e69d3f177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:28.511186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 992/1991] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:28 2022] DELETE /identity/v3/projects/1bec2f81292446bfa3246b679ea82e7c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:28.514317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d21c-2b70-4789-bf51-d57285e01445 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.515326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d21c-2b70-4789-bf51-d57285e01445 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.515611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199d21c-2b70-4789-bf51-d57285e01445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.531202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4941-9759-4394-a0d2-6b9b1321804f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.548810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8199d21c-2b70-4789-bf51-d57285e01445 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:28.550109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8199d21c-2b70-4789-bf51-d57285e01445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.578575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:28.655257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.656115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.656704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.657016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.658405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.660354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.694285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1000/1992] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:28.699366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.710642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.731015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3b33-583b-46e6-b908-799e88f38bae admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.741734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.749728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 993/1993] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:28.749966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.749966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.749966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.749966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.749966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.752344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.752826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.753438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.778637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1001/1994] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:28.781717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.801527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034597df-6657-49eb-9811-490e6c8ce6d4 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.805283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:28.807601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.808617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.810102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.810102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.810472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.811849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.815514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.823470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 994/1995] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:28.824979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.829714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1002/1996] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:28.833818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.836220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4b12-1828-43fd-ae07-b7d3337855c9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.841336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.841864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.842179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.842434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.843117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:28.843418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.845123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.846274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cb6cc-93f9-4175-af1f-b33b3fce36c8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.852094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.852768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.853036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.853620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] PATH_INFO: `/v3/projects/797bba7315f942c987d4117271c6a13b/users/1130ac61acfc4d6fa431258c857b17d8/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.859162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 995/1997] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:50:28.863741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.879606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdffddd-98bf-44e9-970a-c5b197f5cf69 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:28.880241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] RBAC: Authorizing `identity:create_grant(project_id=797bba7315f942c987d4117271c6a13b, user_id=1130ac61acfc4d6fa431258c857b17d8, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.883024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3673b0-1767-41f7-9ca7-577b0217752c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.885147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:28.885824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.886108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.886379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.887185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:28.889165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:28.903116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1003/1998] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:28 2022] PUT /identity/v3/projects/797bba7315f942c987d4117271c6a13b/users/1130ac61acfc4d6fa431258c857b17d8/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:28.917334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e731-b906-4884-a053-da777e941504 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:28.917567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e731-b906-4884-a053-da777e941504 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:28.917763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e731-b906-4884-a053-da777e941504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:28.937191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 996/1999] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:28.938195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:28.961047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1b85e731-b906-4884-a053-da777e941504 None None] MFA Rules not processed for user `1130ac61acfc4d6fa431258c857b17d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:28.962482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1b85e731-b906-4884-a053-da777e941504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:28.981869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:29.023505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fc303-7792-495d-97f5-7d0acc974345 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.031515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.032211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.032490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.032760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.033666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.036371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.068055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 997/2000] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:29.074893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.087234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6e97f-d178-4aeb-a0d5-b62b62bdfdd2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.093445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.094409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.094965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.095338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.096114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.098200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.098584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1004/2001] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:29.111351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85e731-b906-4884-a053-da777e941504 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.113652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1b85e731-b906-4884-a053-da777e941504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:29.134445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 998/2002] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:29.138087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.152492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a23cb-221c-4c42-85cf-61e012e621c2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.171207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.171724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.172010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.172471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.172974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.175184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.199112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 999/2003] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:29.200682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85e731-b906-4884-a053-da777e941504 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.204686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.204836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.205445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.205791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.206161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.216973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.218498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3c212-a485-43fc-aae4-2381f31268f1 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.221910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439048-cd5e-408c-8b58-c3dadcd6d21f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.222737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:29.236023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1005/2004] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:29.239307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.240434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.240773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.241112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] PATH_INFO: `/v3/projects/ab7b4b66e1ad4f2e834dc3f28a911ca9/users/b12b5a162ebe47ebb99151eb7d62b522/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.261136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b4b66e1ad4f2e834dc3f28a911ca9, user_id=b12b5a162ebe47ebb99151eb7d62b522, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.264031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.285968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1000/2005] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:29 2022] PUT /identity/v3/projects/ab7b4b66e1ad4f2e834dc3f28a911ca9/users/b12b5a162ebe47ebb99151eb7d62b522/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:29.298405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.298829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.299212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.331090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] MFA Rules not processed for user `b12b5a162ebe47ebb99151eb7d62b522`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:29.332688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:29.406484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1006/2006] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:29.411265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.476736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa0b5f-06c7-4d52-85b0-0659620bc7ed admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.482426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.483117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.483355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.483634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.484356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.486241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.521871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1001/2007] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:29 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:29.525743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.536301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340b218-4ebc-4065-90f2-f69a58446a23 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.598707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.599472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.599818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.600184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.600830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.602769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.627949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1007/2008] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:29 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:29.632458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.644541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e5b6-9679-45c3-b4b7-31c475c416d1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.651118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.651730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.652013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.652283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.652931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.654555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.667454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1002/2009] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:29.671724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.682422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3656d05-156b-48b0-90fc-dd7b09b5bf69 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.687934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.688760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.689125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.689428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] PATH_INFO: `/v3/projects/d3fb3ed974e14753b789f21f8af59707/users/aeaa321718fb48ecafd6958ac9ad957a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.705635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3fb3ed974e14753b789f21f8af59707, user_id=aeaa321718fb48ecafd6958ac9ad957a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.708440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.721947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1008/2010] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:29 2022] PUT /identity/v3/projects/d3fb3ed974e14753b789f21f8af59707/users/aeaa321718fb48ecafd6958ac9ad957a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:29.726081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.738239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370b4a5-105b-437f-8a4a-f64dacc1f5a0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.788839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:29.800687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.801399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.801821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.802091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.802754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.804463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1d6b7-48c9-4280-b2a7-a965c855c26f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.814942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1003/2011] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:29.819125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:29.830461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182604-689c-4752-b27e-49946390e4b1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:29.835765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:29.836566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.836871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.837209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] PATH_INFO: `/v3/projects/d3fb3ed974e14753b789f21f8af59707/users/aeaa321718fb48ecafd6958ac9ad957a/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.839269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3fb3ed974e14753b789f21f8af59707, user_id=aeaa321718fb48ecafd6958ac9ad957a, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:29.842052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:29.853416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1009/2012] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:29 2022] PUT /identity/v3/projects/d3fb3ed974e14753b789f21f8af59707/users/aeaa321718fb48ecafd6958ac9ad957a/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:29.864636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:29.864954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:29.865235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:29.895612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] MFA Rules not processed for user `aeaa321718fb48ecafd6958ac9ad957a`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:29.896850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:29.968622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1004/2013] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:29.980168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:30.028691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:30.049403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057ddd-6deb-4a5b-89b6-bfe6a9fd26a5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:30.053140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:30.053140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:30.053140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:30.053140 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:30.061383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:30.062858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d47a96-d041-4158-9194-80194c8ec257 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:30.072488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1010/2014] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:30.551644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:30.625676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529789c-57fc-41ee-8236-8fefd6a029b8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:30.630684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:30.631468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:30.631842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:30.632213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] PATH_INFO: `/v3/users/b12b5a162ebe47ebb99151eb7d62b522` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:30.633444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12b5a162ebe47ebb99151eb7d62b522)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:30.635542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:30.691236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12b5a162ebe47ebb99151eb7d62b522'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:30.691918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12b5a162ebe47ebb99151eb7d62b522'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:30.706760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12b5a162ebe47ebb99151eb7d62b522'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:30.714037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12b5a162ebe47ebb99151eb7d62b522'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:30.714871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12b5a162ebe47ebb99151eb7d62b522'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:30.716795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:30.732865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1005/2015] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:30 2022] DELETE /identity/v3/users/b12b5a162ebe47ebb99151eb7d62b522 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:30.738306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:30.739107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:30.739391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:30.767817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:30.769376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:30.832967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1011/2016] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:30.842684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:30.844121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:30.909290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9444-62e0-421a-92cb-99c127140539 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:30.917555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:30.918156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:30.918439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:30.918716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:30.932403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:30.933860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:30.944450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1006/2017] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:31.271887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:31.273566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:31.286533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a03a31-d7f4-4d7a-9bdd-162883a2252a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:31.293604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:31.294273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:31.294557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:31.294861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] PATH_INFO: `/v3/projects/ab7b4b66e1ad4f2e834dc3f28a911ca9` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:31.296167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7b4b66e1ad4f2e834dc3f28a911ca9)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:31.298168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:31.347658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7b4b66e1ad4f2e834dc3f28a911ca9', 'request_id': 'req-c195bb44-54a3-4aca-b4a7-5dbe756699bc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.355947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7b4b66e1ad4f2e834dc3f28a911ca9', 'request_id': 'req-c195bb44-54a3-4aca-b4a7-5dbe756699bc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.362210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7b4b66e1ad4f2e834dc3f28a911ca9', 'request_id': 'req-c195bb44-54a3-4aca-b4a7-5dbe756699bc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.373933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] The token cache is being invalidate because project ab7b4b66e1ad4f2e834dc3f28a911ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:31.374224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.374973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.376554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1012/2018] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:31 2022] DELETE /identity/v3/projects/ab7b4b66e1ad4f2e834dc3f28a911ca9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:31.380759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:31.390903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7458c84-37bd-4a3f-b134-513b862b1e0b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:31.453690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:31.454546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:31.454957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:31.455307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] PATH_INFO: `/v3/users/aeaa321718fb48ecafd6958ac9ad957a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:31.456600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeaa321718fb48ecafd6958ac9ad957a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:31.458872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:31.518688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeaa321718fb48ecafd6958ac9ad957a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.519454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeaa321718fb48ecafd6958ac9ad957a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.534792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeaa321718fb48ecafd6958ac9ad957a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.542771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeaa321718fb48ecafd6958ac9ad957a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.543522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a26d3d52-8811-454e-8222-120c66bb98a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeaa321718fb48ecafd6958ac9ad957a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:31.560868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1007/2019] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:31 2022] DELETE /identity/v3/users/aeaa321718fb48ecafd6958ac9ad957a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:31.997210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:31.998676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:32.011818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bb44-54a3-4aca-b4a7-5dbe756699bc admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.069376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.069960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.070268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.070537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] PATH_INFO: `/v3/projects/d3fb3ed974e14753b789f21f8af59707` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.071732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3fb3ed974e14753b789f21f8af59707)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.073648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.081992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:32.125397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3fb3ed974e14753b789f21f8af59707', 'request_id': 'req-4d684a22-67d2-4907-bf12-ee41de932eb0'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:32.133424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3fb3ed974e14753b789f21f8af59707', 'request_id': 'req-4d684a22-67d2-4907-bf12-ee41de932eb0'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:32.139300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3fb3ed974e14753b789f21f8af59707', 'request_id': 'req-4d684a22-67d2-4907-bf12-ee41de932eb0'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:32.150843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] The token cache is being invalidate because project d3fb3ed974e14753b789f21f8af59707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:32.151195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:32.151827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4d684a22-67d2-4907-bf12-ee41de932eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:32.153340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1013/2020] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:31 2022] DELETE /identity/v3/projects/d3fb3ed974e14753b789f21f8af59707 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:32.158417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.158698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.159013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.181899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:32.183181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:32.195564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:32.274988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1008/2021] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:32.277749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.278516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.279008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.315661 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:32.317004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:32.330973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1014/2022] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:32.335888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.359713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e4943-f61f-4991-b47d-564c60bb0b1c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.366356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.366356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.366356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.366356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.367038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:32.367406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.369100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.385936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1009/2023] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:32.387099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.396522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcbd72-f77c-4965-b8a5-fdbe5883b7e8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.401471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.402081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.402357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.402723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.403382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.405326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.448703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1015/2024] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:32 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:32.452665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.464121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9fc6f-b9e9-4641-a140-8939cc27281c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.528130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.528914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.529201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.529537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.530179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.532282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.560034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1010/2025] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:32.564344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.576612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b473e1c-2df9-4e39-ab28-75b848e014fb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.582768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.583538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.583888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.584217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.585179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.587925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.608404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1016/2026] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:32.613022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.626535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ab3ed-b967-4b90-aaf3-54f2560d188a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.633167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.634753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.635150 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.636094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.637011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.639098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344562-fca5-4d7b-a38d-19adf97f0b6e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.652602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1011/2027] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:32.657372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.668019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29daee6-5728-40c5-8980-f6999f0c5ac5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.675503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.676459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.676892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.677305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] PATH_INFO: `/v3/projects/c21018bfb49b4651ae57b5ea051a57a2/users/642ceda1b66548eca88b24c0100cc2dc/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.697656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21018bfb49b4651ae57b5ea051a57a2, user_id=642ceda1b66548eca88b24c0100cc2dc, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.700827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.717699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1017/2028] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:32 2022] PUT /identity/v3/projects/c21018bfb49b4651ae57b5ea051a57a2/users/642ceda1b66548eca88b24c0100cc2dc/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:32.728314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.728596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.728885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.764159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] MFA Rules not processed for user `642ceda1b66548eca88b24c0100cc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:32.765732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:32.836168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1012/2029] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:32.849541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.851514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:32.911733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0c85-456a-4301-ac7b-22ca455ecbe0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:32.915634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:32.916255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:32.916584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:32.916860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:32.925679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:32.927148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a9e1f-2838-4f0c-bedb-87d15e3282be service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:32.936714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1018/2030] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:32.958602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:32.973525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:33.036736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb951997-3e0d-48fd-8890-adaa8b300d8c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:33.041569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:33.042228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.042508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.042776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] PATH_INFO: `/v3/users/642ceda1b66548eca88b24c0100cc2dc` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:33.044115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642ceda1b66548eca88b24c0100cc2dc)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:33.046110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:33.101179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642ceda1b66548eca88b24c0100cc2dc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.101829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642ceda1b66548eca88b24c0100cc2dc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.115075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642ceda1b66548eca88b24c0100cc2dc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.125993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642ceda1b66548eca88b24c0100cc2dc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.126815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642ceda1b66548eca88b24c0100cc2dc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.145441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1013/2031] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:32 2022] DELETE /identity/v3/users/642ceda1b66548eca88b24c0100cc2dc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:33.147187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.147656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.147921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:33.167697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:33.168902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:33.199195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:33.233484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1019/2032] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:33.240795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:33.242211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:33.300756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d079ec-026b-4fa4-980b-fd57ff3b8606 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:33.304482 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:33.305097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.305369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.305635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:33.314890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:33.316334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0295f-af70-406c-8e4b-c48e9bc82ffc service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:33.325733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1014/2033] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:33.645017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:33.646457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:33.665618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497acc8c-2827-46cd-bf38-e8b35ea5e366 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:33.671624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:33.684992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.685333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.686547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] PATH_INFO: `/v3/projects/c21018bfb49b4651ae57b5ea051a57a2` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:33.688572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21018bfb49b4651ae57b5ea051a57a2)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:33.690680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:33.747097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21018bfb49b4651ae57b5ea051a57a2', 'request_id': 'req-13d1f936-df32-49dd-91ff-b26c27e035fb'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.760114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21018bfb49b4651ae57b5ea051a57a2', 'request_id': 'req-13d1f936-df32-49dd-91ff-b26c27e035fb'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.770517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21018bfb49b4651ae57b5ea051a57a2', 'request_id': 'req-13d1f936-df32-49dd-91ff-b26c27e035fb'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.784391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] The token cache is being invalidate because project c21018bfb49b4651ae57b5ea051a57a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:33.784997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.785828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-13d1f936-df32-49dd-91ff-b26c27e035fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:33.787816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1020/2034] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:33 2022] DELETE /identity/v3/projects/c21018bfb49b4651ae57b5ea051a57a2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:33.792653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.793182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.793366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:33.815710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:33.817000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:33.846961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:33.972231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1015/2035] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:33.986230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:33.986708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:33.986886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.039791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:34.041091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:34.061567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1021/2036] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:34.065868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.081002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7043a-1bd8-4456-b81a-ed3666fe9210 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.092265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.093156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.093588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.093919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.094606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:34.094932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.097559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.118843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1016/2037] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:34.122755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.136080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455c1da-63c4-4434-b66d-5760eceb4d48 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.141287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.141934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.142236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.142510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.143252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.145263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.180334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1022/2038] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:34 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:34.184752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.196299 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083089f-08cb-40b4-84e5-890ad7df0949 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.285351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.285968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.286236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.286511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.287194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.289103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.314118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1017/2039] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:34 2022] POST /identity/v3/users => generated 280 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:34.318299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.328809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22251f-3f9f-4ce4-9e07-77a378246232 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.333627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.334241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.334520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.334787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.335462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.337064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.349303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1023/2040] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:34.353269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.364714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08b18-f53a-4f46-b4df-22acb768e631 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.369453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.370061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.370338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.370611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.371296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.373015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccf187-47c9-4690-80dd-715e329d4662 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.384436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1018/2041] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:34.388631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.399552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1240f-fc78-45ca-8c97-aded5bb7c65b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.405581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.406248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.406552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.406788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] PATH_INFO: `/v3/projects/399b3f9ac6d6407c9046f2cfe953f98e/users/afd2ebc82a624f27a4f08ca66f7fa6a4/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.424572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399b3f9ac6d6407c9046f2cfe953f98e, user_id=afd2ebc82a624f27a4f08ca66f7fa6a4, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.427760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.458950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1024/2042] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:34 2022] PUT /identity/v3/projects/399b3f9ac6d6407c9046f2cfe953f98e/users/afd2ebc82a624f27a4f08ca66f7fa6a4/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:34.470968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.470968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.470968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.509062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] MFA Rules not processed for user `afd2ebc82a624f27a4f08ca66f7fa6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:34.510437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:34.610174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1019/2043] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:34.654078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.655783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:34.735346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dd032-c188-46db-bb57-01d38ee7af11 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.739456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.740341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.741360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.741867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.756355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.759114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec128dc-f4dc-4d74-af72-0d0e435bdb3f service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:34.764754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:34.774365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1025/2044] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:34.793277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:34.834569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:34.916372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581592c-3b29-455d-82cd-fa3e81df0886 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:34.924189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:34.928418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:34.929070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:34.929636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] PATH_INFO: `/v3/users/afd2ebc82a624f27a4f08ca66f7fa6a4` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:34.932157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd2ebc82a624f27a4f08ca66f7fa6a4)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:34.935721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:35.050043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd2ebc82a624f27a4f08ca66f7fa6a4'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.051480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd2ebc82a624f27a4f08ca66f7fa6a4'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.077805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd2ebc82a624f27a4f08ca66f7fa6a4'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.090102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd2ebc82a624f27a4f08ca66f7fa6a4'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.096448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd2ebc82a624f27a4f08ca66f7fa6a4'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.134160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1020/2045] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:34 2022] DELETE /identity/v3/users/afd2ebc82a624f27a4f08ca66f7fa6a4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:35.135952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.136471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.136911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.176680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:35.178036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.266393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1026/2046] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:35.278611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:35.280346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.360403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495f2a2-fa51-4f40-acff-cb2a8462ae62 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:35.368646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:35.369818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.370139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.370409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.388503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:35.390888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674f93-d8aa-43fe-af6e-f6f78937981a service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:35.405249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1021/2047] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:35.708664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:35.710126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.722794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e207136-d0e4-4e2b-961f-3c29ce90e090 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:35.728546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:35.729210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.729457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.729728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] PATH_INFO: `/v3/projects/399b3f9ac6d6407c9046f2cfe953f98e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.731044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=399b3f9ac6d6407c9046f2cfe953f98e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:35.733510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:35.790824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da09b-e939-41d3-93ec-0ea53c30ada5 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.791033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da09b-e939-41d3-93ec-0ea53c30ada5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.791314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da09b-e939-41d3-93ec-0ea53c30ada5 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.792698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1022/2048] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:50:35.796413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399b3f9ac6d6407c9046f2cfe953f98e', 'request_id': 'req-e92cba1d-fd0a-4df3-8646-af947eaa05ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.804501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399b3f9ac6d6407c9046f2cfe953f98e', 'request_id': 'req-e92cba1d-fd0a-4df3-8646-af947eaa05ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.810121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399b3f9ac6d6407c9046f2cfe953f98e', 'request_id': 'req-e92cba1d-fd0a-4df3-8646-af947eaa05ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.821350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d844-8e0b-4fa4-adbd-11b54b393d89 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.821890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] The token cache is being invalidate because project 399b3f9ac6d6407c9046f2cfe953f98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:35.821890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.822421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d844-8e0b-4fa4-adbd-11b54b393d89 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.822699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d844-8e0b-4fa4-adbd-11b54b393d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.822826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e92cba1d-fd0a-4df3-8646-af947eaa05ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:35.824573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1027/2049] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:35 2022] DELETE /identity/v3/projects/399b3f9ac6d6407c9046f2cfe953f98e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:35.829097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.829404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.829748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.850007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2786d844-8e0b-4fa4-adbd-11b54b393d89 None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:35.852454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:35.852631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2786d844-8e0b-4fa4-adbd-11b54b393d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.853853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.902919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:35.918516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1028/2050] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:35.924948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:35.924948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.926114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1023/2051] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:50:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:35.929401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.929693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.929967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.936252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc80d5e-9efa-4e05-b636-7d8eda8e9872 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:35.941107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:35.941746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.942027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.942297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] PATH_INFO: `/v3/users/a959dec6c0c944aea231b65612892705` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:35.943454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a959dec6c0c944aea231b65612892705)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:35.945391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:35.950982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:35.952186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.960117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1024/2052] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:35.960832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:35.962264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:35.965755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:35.990341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d6e3-12a7-406d-a732-d3694759d847 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:35.997566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:35.998192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:35.998479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:35.998764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.006127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a959dec6c0c944aea231b65612892705'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.010691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.012392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.018498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a959dec6c0c944aea231b65612892705'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.025670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1025/2053] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:36.025825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a959dec6c0c944aea231b65612892705'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.026407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a959dec6c0c944aea231b65612892705'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.027056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a959dec6c0c944aea231b65612892705'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.027321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.040779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbf27-26eb-4ad3-8679-19ef3691ccf7 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.043087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1029/2054] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:35 2022] DELETE /identity/v3/users/a959dec6c0c944aea231b65612892705 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:36.127057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.129007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.129502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.129954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.130996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:36.131812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.134376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.171546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1026/2055] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:36.171815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.173453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.189657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e526c5-60b7-4854-bac4-3d0b9bbc3e2b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.195293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.195933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.196216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.196490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.197280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.201430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.241144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1030/2056] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:36.246237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.259154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd590e-1d7e-4552-934a-8b210e2d0425 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.348632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.349254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.349528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.349794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.350431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.352573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.380703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1027/2057] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:36.381871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.396476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44a40-4341-4aff-b6bc-16c0430f9e47 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.404585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.405340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.405546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.405822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.406918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.409266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.433038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.433289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1031/2058] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:36.444976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37925-ac37-4d22-9f01-c3f01538126d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.450563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.451253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.451524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.451790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.452435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.454042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce3b82-b186-4e66-b56f-8f96f5045792 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.466845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1028/2059] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:36.470243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.487499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887aedc-3826-46bb-90a0-f890b71a4cd9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.495069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.496247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.496247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.496656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] PATH_INFO: `/v3/projects/5ff385f1666d49deafb23270d0d973f1/users/6b3e363ea1484b3597a95985f3cc8ee3/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.515531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff385f1666d49deafb23270d0d973f1, user_id=6b3e363ea1484b3597a95985f3cc8ee3, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.524380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.538780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1032/2060] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:36 2022] PUT /identity/v3/projects/5ff385f1666d49deafb23270d0d973f1/users/6b3e363ea1484b3597a95985f3cc8ee3/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:36.549663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.549979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.550481 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.577211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.583567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] MFA Rules not processed for user `6b3e363ea1484b3597a95985f3cc8ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:36.583871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.585293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.675727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:36.724332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1029/2061] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:36.727033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfafb8-f55f-4062-bdf8-c56895b6ee8e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.728918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.732648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.733314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.733589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.733852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] PATH_INFO: `/v3/projects/968cc9376f884f67b20b91547211fffd` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.735011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] RBAC: Authorizing `identity:delete_project(project_id=968cc9376f884f67b20b91547211fffd)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.736999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.740804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d517e1f-54d8-4c66-8605-3e63833b2e98 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.746333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.746931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.747256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.747536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.748330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.750496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.787070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1030/2062] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:36.788290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:36.788547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.798654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968cc9376f884f67b20b91547211fffd', 'request_id': 'req-1b776109-f601-4b30-a552-4620d116220d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.808245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968cc9376f884f67b20b91547211fffd', 'request_id': 'req-1b776109-f601-4b30-a552-4620d116220d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.814758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968cc9376f884f67b20b91547211fffd', 'request_id': 'req-1b776109-f601-4b30-a552-4620d116220d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.826994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] The token cache is being invalidate because project 968cc9376f884f67b20b91547211fffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:36.827360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.828095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-1b776109-f601-4b30-a552-4620d116220d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:36.829677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1033/2063] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:36 2022] DELETE /identity/v3/projects/968cc9376f884f67b20b91547211fffd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:36.839989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ca4d-34bb-4490-ac34-785eb1cd546a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.840280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ca4d-34bb-4490-ac34-785eb1cd546a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.840557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ca4d-34bb-4490-ac34-785eb1cd546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.854309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b1f83-5b88-4af2-8e7e-4fb4862aa259 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.859344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.859948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.860219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.860484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.860970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-31e3ca4d-34bb-4490-ac34-785eb1cd546a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:36.861108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.862243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-31e3ca4d-34bb-4490-ac34-785eb1cd546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.863110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.870360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1034/2064] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:36.878547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.879049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.879306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.886005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1031/2065] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:36.889610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.891305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.901105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:36.902272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:36.903349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76803b9-b223-407b-93d1-24daf9f94afa admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.907986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.908591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.908865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.909121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.909753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.911370 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.911479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1035/2066] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:36.915609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.923429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1032/2067] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:36.924854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562d63f-a170-425a-a34a-cdf1b4c39657 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.927292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.929592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.930185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.930456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.930714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.931395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:36.931690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.933310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.937077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1e24-2614-4a76-a72a-00fac5db115a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.941688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.942316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.942589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.942882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.943512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.945046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1036/2068] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:50:36.945141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.949226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.957072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1033/2069] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:36.959531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b22-6d09-40a8-95c9-a7f37979c0ef admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.961250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:36.964479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.965069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.965341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.965611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.966220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.968314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.971315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b881d0-d2ea-4b81-a879-3071eea867f7 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:36.976141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:36.976774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:36.977045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:36.977313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] PATH_INFO: `/v3/projects/7f423eb73737444daaa95c7e79ba133f/users/5d5f1a139b0d46c1b7171f7a901422fe/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:36.992333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f423eb73737444daaa95c7e79ba133f, user_id=5d5f1a139b0d46c1b7171f7a901422fe, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:36.994902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de5768-d4f2-4d24-97ae-669976c1473b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:36.999252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1037/2070] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:37.000623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.008442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1034/2071] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:36 2022] PUT /identity/v3/projects/7f423eb73737444daaa95c7e79ba133f/users/5d5f1a139b0d46c1b7171f7a901422fe/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:37.067074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002ca0-aa48-4887-9029-4b5cf03792b9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.077556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.077834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.078126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.104927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] MFA Rules not processed for user `5d5f1a139b0d46c1b7171f7a901422fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:37.106119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.123870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.124481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.124755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.125029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.125690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.127654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.145489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:37.154601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1038/2072] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:36 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:37.155652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.165210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc128e-338e-41b8-b9ee-8af34b996542 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.169751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.170393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.170599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.170892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.171547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.173079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1035/2073] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:37.174844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.178843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.187078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1039/2074] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:37.191364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.191923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550b61-49ca-4ecd-8ca2-225cf9806ca4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.197987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.199021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:37.199800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.199800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.200206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.201139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.203100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5559a-6034-4810-b40b-f53b14d79595 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.203245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.210190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.210832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.211143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.211430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.212266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.215261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.228136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1040/2075] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:37.232576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.237157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1036/2076] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:37.240726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.243275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f2a88-947a-4320-acf7-945a1f0f835f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.309958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.310634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.311554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.311835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] PATH_INFO: `/v3/projects/3e26ed3d4dab4d29972e285ce7063abd/users/603363772e4d4a1992bc534fa3cbd69e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.315113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e25a32-9099-410a-b659-4e536931c439 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.320658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.321306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.321585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.321857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.322548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.324573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.327408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e26ed3d4dab4d29972e285ce7063abd, user_id=603363772e4d4a1992bc534fa3cbd69e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.330074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc3dba-f519-43df-ad2b-f371c03005fd admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.345255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1041/2077] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:37 2022] PUT /identity/v3/projects/3e26ed3d4dab4d29972e285ce7063abd/users/603363772e4d4a1992bc534fa3cbd69e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:37.347216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1037/2078] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:37.356663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.356948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.357674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.357776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.372588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe023-20bb-4216-babb-3cc4f4cd9f3f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.387542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] MFA Rules not processed for user `603363772e4d4a1992bc534fa3cbd69e`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:37.388778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.429110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.429783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.430119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.430365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.431041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.432670 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.442946 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1038/2079] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:37.447004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.455162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1042/2080] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:37.457512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be65580-6945-41a4-9737-f3d32f3a4239 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.459912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.462378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.463052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.463444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.463722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] PATH_INFO: `/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3/users/bbefeba171384a1298d384210c41739e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.470186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b347a-04a7-4fc0-bfa5-079cd652061b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.474820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.475448 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.475724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.475990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.476634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.478416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aeabb9c1ef94446ac0e89ed911807d3, user_id=bbefeba171384a1298d384210c41739e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.478534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.481100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.497235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1039/2081] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:37 2022] PUT /identity/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3/users/bbefeba171384a1298d384210c41739e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:37.501334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.506239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1043/2082] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:37.510024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.520573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:37.555784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:37.564630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022365cc-2c8f-4aa1-bc64-29247c5be1ad admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.574161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333165-f39c-4943-a725-11db65c486a5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.579148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.579867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.580182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.580461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.581093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.583171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.608340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1044/2083] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:50:37.609627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.619236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7a2af-02c5-434f-8283-d10d426daad0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.620140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.620748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.621026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.621290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.621930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.623681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.624175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.624928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.625217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.625839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.626485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.628128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.633991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1040/2084] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:37.638083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.647008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1045/2085] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:50:37.648012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.651697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa797-67f8-4de4-bd1d-8815c27631ec admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.657882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0dcb-8e00-4dfa-a1ee-d222cd0c083d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.660211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.660863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.661134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.661399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] PATH_INFO: `/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3/users/bbefeba171384a1298d384210c41739e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.663109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.663318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aeabb9c1ef94446ac0e89ed911807d3, user_id=bbefeba171384a1298d384210c41739e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.664262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.664637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.665018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] PATH_INFO: `/v3/projects/cd625578db0748d893302c9d84a0e550/users/a86a05ae9195411a819bd40cd88a5aa5/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.665972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52629187-3e6d-4c28-8753-09fdabd212eb admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.677879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1041/2086] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:37 2022] PUT /identity/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3/users/bbefeba171384a1298d384210c41739e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:37.690999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd625578db0748d893302c9d84a0e550, user_id=a86a05ae9195411a819bd40cd88a5aa5, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.692400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.692680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.692950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.693653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.723807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1046/2087] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:37 2022] PUT /identity/v3/projects/cd625578db0748d893302c9d84a0e550/users/a86a05ae9195411a819bd40cd88a5aa5/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:37.727581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.728250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] MFA Rules not processed for user `bbefeba171384a1298d384210c41739e`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:37.729449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.804171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ffadb-f939-43c8-aa86-6170f3e4cd1c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.809412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1042/2088] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:37.809557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.810144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.810431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.810708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.811390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.813978 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.824458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.825316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1047/2089] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:37.828121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.829181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.840313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a1d9-8fe1-4611-bef4-fba1e32a59ce admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.845459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.846156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.846433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.846705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] PATH_INFO: `/v3/projects/cd625578db0748d893302c9d84a0e550/users/a86a05ae9195411a819bd40cd88a5aa5/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.849886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd625578db0748d893302c9d84a0e550, user_id=a86a05ae9195411a819bd40cd88a5aa5, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:37.851526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d443a7-f002-4c9c-915a-e050e338f1b3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:37.863229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1048/2090] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:37 2022] PUT /identity/v3/projects/cd625578db0748d893302c9d84a0e550/users/a86a05ae9195411a819bd40cd88a5aa5/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:37.866855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:37.876118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.876626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.877128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.893387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01196d-c1a6-4475-bfe8-552643e90090 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:37.906722 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] MFA Rules not processed for user `a86a05ae9195411a819bd40cd88a5aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:37.907984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.949275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:37.949955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:37.950241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:37.950515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:37.952606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:37.985985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1049/2091] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:37.997461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:37.999071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:38.011273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc8b92-edd9-4df0-98f5-f34f1e718ef2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:38.015046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:38.015752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:38.015918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:38.016218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:38.022369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:38.023870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:38.025501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:38.026411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:38.028486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431076-1d51-453e-b31a-3b404249b87b service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:38.034638 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1043/2092] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:38.040306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1050/2093] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:39.702012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:39.768091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bb0ff-f0cb-4dd6-923b-01b11a107008 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:39.772797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:39.773452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:39.773765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:39.774039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] PATH_INFO: `/v3/users/603363772e4d4a1992bc534fa3cbd69e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:39.775211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603363772e4d4a1992bc534fa3cbd69e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:39.777162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:39.823095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:39.824993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603363772e4d4a1992bc534fa3cbd69e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:39.825909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603363772e4d4a1992bc534fa3cbd69e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:39.842540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603363772e4d4a1992bc534fa3cbd69e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:39.849836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603363772e4d4a1992bc534fa3cbd69e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:39.850548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603363772e4d4a1992bc534fa3cbd69e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:39.865422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1044/2094] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:39 2022] DELETE /identity/v3/users/603363772e4d4a1992bc534fa3cbd69e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:39.870002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:39.870602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:39.870866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:39.890383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:39.891617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:39.948452 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1051/2095] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:39.959619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:39.961104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:40.022197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623283b7-c0dc-4427-8826-251d2499ebe5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:40.026018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:40.026739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:40.027099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:40.027433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:40.036902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:40.038405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:40.048403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1045/2096] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:40.521531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:40.523161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:40.566655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e022ce-6c74-4bb3-8dcc-4cc9573b3355 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:40.575182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:40.576007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:40.577303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:40.577632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] PATH_INFO: `/v3/projects/3e26ed3d4dab4d29972e285ce7063abd` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:40.579517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e26ed3d4dab4d29972e285ce7063abd)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:40.581953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:40.656478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e26ed3d4dab4d29972e285ce7063abd', 'request_id': 'req-d7c61f73-0a48-4f7e-bbab-0af09d75420f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.680384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e26ed3d4dab4d29972e285ce7063abd', 'request_id': 'req-d7c61f73-0a48-4f7e-bbab-0af09d75420f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.696152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e26ed3d4dab4d29972e285ce7063abd', 'request_id': 'req-d7c61f73-0a48-4f7e-bbab-0af09d75420f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.714712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] The token cache is being invalidate because project 3e26ed3d4dab4d29972e285ce7063abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:40.715049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.715772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.717889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1052/2097] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:40 2022] DELETE /identity/v3/projects/3e26ed3d4dab4d29972e285ce7063abd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:40.726293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:40.735824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794191fa-84b1-4601-a052-7b07fc48eded service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:40.764275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:40.766229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:40.885387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:40.887758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:40.888223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:40.888637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] PATH_INFO: `/v3/users/a86a05ae9195411a819bd40cd88a5aa5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:40.889805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86a05ae9195411a819bd40cd88a5aa5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:40.892104 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:40.896454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61f73-0a48-4f7e-bbab-0af09d75420f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:40.919094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:40.919317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:40.919741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:40.919865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:40.922951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:40.994864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86a05ae9195411a819bd40cd88a5aa5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.994864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86a05ae9195411a819bd40cd88a5aa5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:40.995149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:41.006293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86a05ae9195411a819bd40cd88a5aa5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.009068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:41.019115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86a05ae9195411a819bd40cd88a5aa5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.020606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86a05ae9195411a819bd40cd88a5aa5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.053900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:41.055419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b446ff-dd5b-49f1-bcad-615596cba0bb service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:41.061975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1046/2098] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:40 2022] DELETE /identity/v3/users/a86a05ae9195411a819bd40cd88a5aa5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:41.070453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1053/2099] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:41.525862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:41.527514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:41.675537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60009b-0e09-4ac7-bbde-25fbf69c4082 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:41.680719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:41.680937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.681216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.681483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] PATH_INFO: `/v3/projects/cd625578db0748d893302c9d84a0e550` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.682727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd625578db0748d893302c9d84a0e550)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:41.691356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:41.740749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd625578db0748d893302c9d84a0e550', 'request_id': 'req-2ea5446c-178e-4987-9c91-d33e5f38c5a1'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.750365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd625578db0748d893302c9d84a0e550', 'request_id': 'req-2ea5446c-178e-4987-9c91-d33e5f38c5a1'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.760550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd625578db0748d893302c9d84a0e550', 'request_id': 'req-2ea5446c-178e-4987-9c91-d33e5f38c5a1'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.772197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] The token cache is being invalidate because project cd625578db0748d893302c9d84a0e550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:41.772484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.773137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:41.774614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1047/2100] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:41 2022] DELETE /identity/v3/projects/cd625578db0748d893302c9d84a0e550 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:41.779942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7d52-ca89-4ba7-b5c1-338afec2a593 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.780231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7d52-ca89-4ba7-b5c1-338afec2a593 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.780931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7d52-ca89-4ba7-b5c1-338afec2a593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.810741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-95dd7d52-ca89-4ba7-b5c1-338afec2a593 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:41.812064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-95dd7d52-ca89-4ba7-b5c1-338afec2a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:41.839800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:41.842172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:41.893725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1054/2101] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:41.901408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.901693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.901970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.912847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5446c-178e-4987-9c91-d33e5f38c5a1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:41.916556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:41.918843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.918843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.918843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.919907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:41.921952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:41.923161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:41.931851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1055/2102] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:41.935982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:41.945113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ccc53-4fe8-4d00-97fd-c89eeeb4fd0b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:41.950060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:41.950882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.951161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.951427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.952066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:41.952363 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:41.955678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476855db-617e-477d-be5b-243942a34a3a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:41.966224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1056/2103] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:50:41.967143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:41.970466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:41.970829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:41.971136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:41.990977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] MFA Rules not processed for user `658ef1a43f7d445a920e5c4f2b26e339`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:41.992262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:42.006834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.008364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.017967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1048/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:42.019714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.031666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4132-dc7c-492d-9a5f-20fcb3f44a64 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.036962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.037801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.038201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.038573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.039371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.041446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.060325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1057/2105] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:42.074777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1049/2106] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:42.079124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.094781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4be725-47c6-428c-b174-a1fa66794ea5 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.172550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:42.179504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.180124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.180399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.180667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.181318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.183298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.207747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1058/2107] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:42.211717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.223448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f79dd0-af04-47c0-b42e-e54aac3830f2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.229247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.229890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.230640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.230640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.236041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.238449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.257902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1050/2108] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:42.258905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.273557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4d8f-7c21-4e1c-a757-bc73774b074d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.287374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.290498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.290498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.290498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.290498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.292157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c879a7e-3534-41f5-bb3f-0630df4764e8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.308575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1059/2109] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:42.310577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.322655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb4c2-738e-4abc-b059-b0f72b67697a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.328050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.328806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.329143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.329488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] PATH_INFO: `/v3/projects/bcdfce72af814b0aad4690115d582f10/users/96d66a0e2c44486687b0b1dd5d9eb5eb/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.358337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdfce72af814b0aad4690115d582f10, user_id=96d66a0e2c44486687b0b1dd5d9eb5eb, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.358337 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.366105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1051/2110] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:42 2022] PUT /identity/v3/projects/bcdfce72af814b0aad4690115d582f10/users/96d66a0e2c44486687b0b1dd5d9eb5eb/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:42.377562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.377937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.378282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.406627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] MFA Rules not processed for user `96d66a0e2c44486687b0b1dd5d9eb5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:42.407918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:42.474720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1060/2111] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:42.479218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.541233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eff5f9-45b8-44f7-982f-9d6b33a04906 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.545906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.546512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.546786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.547086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.547735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.549691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.578028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1052/2112] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:42.582417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.593746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f34108-1814-47f2-90f2-dfa5c8384c0b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.673058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.674053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.674467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.674886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.675902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.678046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.718856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1061/2113] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:42.719705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.731152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841a66-9e92-40a9-b6bb-459ac2a88981 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.737961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.739153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.739628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.740034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.742277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.744676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.757433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1053/2114] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:42.762171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.779010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253387b-07a6-4fff-affe-4b25aad4bf51 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.784131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.784830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.785135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.785454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] PATH_INFO: `/v3/projects/c141b1517b0f4477a609163912b97a01/users/1851a0935e874e7981b53f1b7a2991bd/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.803924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c141b1517b0f4477a609163912b97a01, user_id=1851a0935e874e7981b53f1b7a2991bd, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.807180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.824760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1062/2115] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:42 2022] PUT /identity/v3/projects/c141b1517b0f4477a609163912b97a01/users/1851a0935e874e7981b53f1b7a2991bd/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:42.829576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.841392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fcf59-a7c5-445e-8efe-30053c116ee9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.888914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:42.907427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.908409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.908754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.909096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.909813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.911522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a8c73-4153-4fe7-bf44-4ecf5fb56f20 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.921802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1054/2116] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:42.926315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:42.938271 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a8474-65d7-4b92-a927-4b4a02ccefab admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:42.944822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:42.945597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.946086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.946431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] PATH_INFO: `/v3/projects/c141b1517b0f4477a609163912b97a01/users/1851a0935e874e7981b53f1b7a2991bd/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:42.948582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c141b1517b0f4477a609163912b97a01, user_id=1851a0935e874e7981b53f1b7a2991bd, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:42.951341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:42.962034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1063/2117] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:42 2022] PUT /identity/v3/projects/c141b1517b0f4477a609163912b97a01/users/1851a0935e874e7981b53f1b7a2991bd/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:42.972686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:42.972973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:42.973279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:43.003682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] MFA Rules not processed for user `1851a0935e874e7981b53f1b7a2991bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:43.004895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:43.078363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1055/2118] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:43.090039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:43.112842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:43.152255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a76d7-b769-4b37-a667-26ddf327a989 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:43.155898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:43.161149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:43.161502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:43.161781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:43.222772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:43.224465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb0615-d700-499d-a7f6-081a5be15f51 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:43.237030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1064/2119] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:46.053388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:46.054836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:46.131567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480a724-3e3b-4a67-a537-59fe80eb5e7d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:46.137499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:46.138144 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:46.138419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:46.138685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] PATH_INFO: `/v3/users/6b3e363ea1484b3597a95985f3cc8ee3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:46.139856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3e363ea1484b3597a95985f3cc8ee3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:46.141784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:46.212405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3e363ea1484b3597a95985f3cc8ee3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.213115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3e363ea1484b3597a95985f3cc8ee3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.239956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3e363ea1484b3597a95985f3cc8ee3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.241589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:46.251385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3e363ea1484b3597a95985f3cc8ee3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.254030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3e363ea1484b3597a95985f3cc8ee3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.270944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1056/2120] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:46 2022] DELETE /identity/v3/users/6b3e363ea1484b3597a95985f3cc8ee3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:46.275502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:46.276051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:46.276051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:46.296455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:46.297665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:46.357741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1065/2121] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:46.367597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:46.369023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:46.440357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6e1b9-dadb-4189-b492-4524f03d8b40 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:46.444727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:46.445336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:46.445610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:46.445881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:46.456056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:46.457496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:46.467253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1057/2122] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:46.775791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:46.777420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:46.795231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a672f0-3215-4d94-bcb0-2141ad93e050 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:46.808483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:46.809447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:46.809789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:46.810015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] PATH_INFO: `/v3/projects/5ff385f1666d49deafb23270d0d973f1` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:46.812073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff385f1666d49deafb23270d0d973f1)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:46.817955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:46.873275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff385f1666d49deafb23270d0d973f1', 'request_id': 'req-6a528f1e-7390-439d-a210-467e783194ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.880444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff385f1666d49deafb23270d0d973f1', 'request_id': 'req-6a528f1e-7390-439d-a210-467e783194ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.886280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff385f1666d49deafb23270d0d973f1', 'request_id': 'req-6a528f1e-7390-439d-a210-467e783194ab'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.888088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:46.899202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] The token cache is being invalidate because project 5ff385f1666d49deafb23270d0d973f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:46.899694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.900634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:46.902159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1066/2123] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:46 2022] DELETE /identity/v3/projects/5ff385f1666d49deafb23270d0d973f1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:46.906372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:46.916793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe62cd-3e3f-4e78-9e9f-bd1e5aacb125 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:46.972116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:46.972763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:46.973032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:46.973296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] PATH_INFO: `/v3/users/5d5f1a139b0d46c1b7171f7a901422fe` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:46.974524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d5f1a139b0d46c1b7171f7a901422fe)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:46.976491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:47.025831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d5f1a139b0d46c1b7171f7a901422fe'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.026630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d5f1a139b0d46c1b7171f7a901422fe'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.039664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d5f1a139b0d46c1b7171f7a901422fe'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.046320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d5f1a139b0d46c1b7171f7a901422fe'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.047088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d5f1a139b0d46c1b7171f7a901422fe'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.060807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1058/2124] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:46 2022] DELETE /identity/v3/users/5d5f1a139b0d46c1b7171f7a901422fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:47.794683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:47.800199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:47.823777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528f1e-7390-439d-a210-467e783194ab admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:47.892496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:47.893252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:47.893530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:47.893937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] PATH_INFO: `/v3/projects/7f423eb73737444daaa95c7e79ba133f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:47.895334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f423eb73737444daaa95c7e79ba133f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:47.897581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:47.950747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f423eb73737444daaa95c7e79ba133f', 'request_id': 'req-9c68f28b-0156-4cde-8a10-7d582c4de500'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.960045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f423eb73737444daaa95c7e79ba133f', 'request_id': 'req-9c68f28b-0156-4cde-8a10-7d582c4de500'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.967040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f423eb73737444daaa95c7e79ba133f', 'request_id': 'req-9c68f28b-0156-4cde-8a10-7d582c4de500'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.976122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:47.977728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:47.979250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] The token cache is being invalidate because project 7f423eb73737444daaa95c7e79ba133f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:47.979542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.980182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:47.981670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1067/2125] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:47 2022] DELETE /identity/v3/projects/7f423eb73737444daaa95c7e79ba133f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:47.985125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:47.986516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:47.987910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:48.055089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9ff3-ab16-4e5c-b2e3-3bd01f7c7f7b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:48.063277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68f28b-0156-4cde-8a10-7d582c4de500 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:48.069113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:48.069963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:48.070413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:48.070878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] PATH_INFO: `/v3/users/bbefeba171384a1298d384210c41739e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:48.077777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbefeba171384a1298d384210c41739e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:48.080000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:48.109415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:48.110073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:48.110288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:48.110556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:48.112386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:48.112549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:48.134117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbefeba171384a1298d384210c41739e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.146185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbefeba171384a1298d384210c41739e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.153880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbefeba171384a1298d384210c41739e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.154619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbefeba171384a1298d384210c41739e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.155268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbefeba171384a1298d384210c41739e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.169265 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1068/2126] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:47 2022] DELETE /identity/v3/users/bbefeba171384a1298d384210c41739e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:48.180579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:48.182181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:48.192294 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1059/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:48.575584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:48.577571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:48.679977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306aa06-e3c5-460f-938a-e5fb1f7a6a39 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:48.684424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:48.685058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:48.685330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:48.685596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] PATH_INFO: `/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:48.686690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aeabb9c1ef94446ac0e89ed911807d3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:48.688631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:48.735874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aeabb9c1ef94446ac0e89ed911807d3', 'request_id': 'req-03305881-c5fe-41e4-a74a-7bebb5ec20e2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.744020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aeabb9c1ef94446ac0e89ed911807d3', 'request_id': 'req-03305881-c5fe-41e4-a74a-7bebb5ec20e2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.750147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aeabb9c1ef94446ac0e89ed911807d3', 'request_id': 'req-03305881-c5fe-41e4-a74a-7bebb5ec20e2'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.761779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] The token cache is being invalidate because project 7aeabb9c1ef94446ac0e89ed911807d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:48.762060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.762695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-03305881-c5fe-41e4-a74a-7bebb5ec20e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:48.764197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1069/2128] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:48 2022] DELETE /identity/v3/projects/7aeabb9c1ef94446ac0e89ed911807d3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:53.203790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:53.205439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:53.219596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb913516-7969-4b00-9510-779d8b12e3db service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:53.285133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:53.285850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:53.286131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:53.286398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] PATH_INFO: `/v3/users/96d66a0e2c44486687b0b1dd5d9eb5eb` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:53.287594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d66a0e2c44486687b0b1dd5d9eb5eb)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:53.289570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:53.302301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:53.337097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d66a0e2c44486687b0b1dd5d9eb5eb'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.337795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d66a0e2c44486687b0b1dd5d9eb5eb'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.350217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d66a0e2c44486687b0b1dd5d9eb5eb'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.357902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d66a0e2c44486687b0b1dd5d9eb5eb'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.358594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d66a0e2c44486687b0b1dd5d9eb5eb'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.373820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1060/2129] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:53 2022] DELETE /identity/v3/users/96d66a0e2c44486687b0b1dd5d9eb5eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:53.378440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:53.378982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:53.379132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:53.401728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:53.403009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:53.435159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:53.465467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1070/2130] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:53.477044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:53.478453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:53.545353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c7b3-d8a2-4e75-abef-8a0e0021ce5e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:53.550743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:53.551395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:53.551672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:53.551936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:53.562330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:53.563804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:53.573647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1061/2131] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:53.875573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:53.877349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:53.891213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e944d-f31e-4ca6-8de4-6d070914b202 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:53.897036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:53.897656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:53.897963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:53.898229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] PATH_INFO: `/v3/projects/bcdfce72af814b0aad4690115d582f10` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:53.899520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdfce72af814b0aad4690115d582f10)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:53.901488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:53.949752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdfce72af814b0aad4690115d582f10', 'request_id': 'req-2920a37b-0b6c-4747-9763-8edabc7a34a9'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.957375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdfce72af814b0aad4690115d582f10', 'request_id': 'req-2920a37b-0b6c-4747-9763-8edabc7a34a9'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.963617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdfce72af814b0aad4690115d582f10', 'request_id': 'req-2920a37b-0b6c-4747-9763-8edabc7a34a9'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.974385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] The token cache is being invalidate because project bcdfce72af814b0aad4690115d582f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:53.974668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.975338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:53.977158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1071/2132] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:53 2022] DELETE /identity/v3/projects/bcdfce72af814b0aad4690115d582f10 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:53.981135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:53.990976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44bc7-f74b-4bb8-aff9-ec618555688b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:54.052375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:54.053058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:54.053332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:54.053597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] PATH_INFO: `/v3/users/1851a0935e874e7981b53f1b7a2991bd` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:54.054737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1851a0935e874e7981b53f1b7a2991bd)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:54.056700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:54.134920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:54.147501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1851a0935e874e7981b53f1b7a2991bd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.148209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1851a0935e874e7981b53f1b7a2991bd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.162656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1851a0935e874e7981b53f1b7a2991bd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.172508 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1851a0935e874e7981b53f1b7a2991bd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.173465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-224fde37-bc51-42c0-93db-58501f298644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1851a0935e874e7981b53f1b7a2991bd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.195837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1062/2133] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:53 2022] DELETE /identity/v3/users/1851a0935e874e7981b53f1b7a2991bd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:54.628807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:54.630847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:54.647166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920a37b-0b6c-4747-9763-8edabc7a34a9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:54.716160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:54.716900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:54.717181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:54.717453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] PATH_INFO: `/v3/projects/c141b1517b0f4477a609163912b97a01` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:54.718637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c141b1517b0f4477a609163912b97a01)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:54.720836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:54.771095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c141b1517b0f4477a609163912b97a01', 'request_id': 'req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.779751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c141b1517b0f4477a609163912b97a01', 'request_id': 'req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.786840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c141b1517b0f4477a609163912b97a01', 'request_id': 'req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.788250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:54.800382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] The token cache is being invalidate because project c141b1517b0f4477a609163912b97a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:54.800667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.801331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0f4cd30-86b1-4819-9ef6-fc853e5c7c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:54.802827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1072/2134] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:54 2022] DELETE /identity/v3/projects/c141b1517b0f4477a609163912b97a01 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:54.810503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:54.811036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:54.811288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:54.834201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:54.835457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:54.900945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1063/2135] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:54.912853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:54.913328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:54.913673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:54.937338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:54.938575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:54.947761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1073/2136] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:54.952248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:54.962491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77507ed-36fd-4f23-97e7-3e2a289033b8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:54.968486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:54.969096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:54.969379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:54.969655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:54.970355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:54.970662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:54.972320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:54.985338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1064/2137] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:54.990052 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.000580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aecb9-cbd8-40ca-8679-3c5d19c3533f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.005791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.006487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.006776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.007082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.007738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.009760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.040841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1074/2138] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:54 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:55.045138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.055818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88db4-5a5b-4439-9e66-9f0c0c4fa0ff admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.122617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.123296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.123582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.123852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.124865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.127059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.154975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1065/2139] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:55 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:55.159051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.170967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a70c5-625b-461d-aa39-cb9dc0a114e9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.176994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.177711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.178029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.178331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.178976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.180561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.196353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1075/2140] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:55.200640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.213177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2558-8f7c-4a54-b963-27a17bad93c3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.218402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.219033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.219310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.219579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.220222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.221928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d23db-c974-4314-b60c-966ff7799f8e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.233668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1066/2141] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:55.239942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.250446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867247d0-a22f-4b95-a497-471b348bfc46 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.255803 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.256470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.256754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.257130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] PATH_INFO: `/v3/projects/b94fc1b5379a42a7923ee23e5043ed9f/users/c76ecf5b37594fd7b99d80cb9889e62e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.277044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94fc1b5379a42a7923ee23e5043ed9f, user_id=c76ecf5b37594fd7b99d80cb9889e62e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.283097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.298600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1076/2142] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:55 2022] PUT /identity/v3/projects/b94fc1b5379a42a7923ee23e5043ed9f/users/c76ecf5b37594fd7b99d80cb9889e62e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:50:55.312169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.312658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.313081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.344851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] MFA Rules not processed for user `c76ecf5b37594fd7b99d80cb9889e62e`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:55.346349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:55.351626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:55.415812 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1067/2143] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:55.428550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.430649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:55.490203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb59d7c-737b-42a6-bd33-88705a960a77 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.494059 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.494715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.495018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.495273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.505081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.506520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d49f44-d01d-4af5-a09e-96c864128fe1 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.516894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1077/2144] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:55.549509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.615849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0dca8-ca78-44e9-8034-4dbf40beb20d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.621397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.622064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.622339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.622637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] PATH_INFO: `/v3/users/c76ecf5b37594fd7b99d80cb9889e62e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.623890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76ecf5b37594fd7b99d80cb9889e62e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.625877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.685439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76ecf5b37594fd7b99d80cb9889e62e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:55.686190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76ecf5b37594fd7b99d80cb9889e62e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:55.699185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76ecf5b37594fd7b99d80cb9889e62e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:55.707122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76ecf5b37594fd7b99d80cb9889e62e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:55.707360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76ecf5b37594fd7b99d80cb9889e62e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:55.725732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1068/2145] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:50:55 2022] DELETE /identity/v3/users/c76ecf5b37594fd7b99d80cb9889e62e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:55.731464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.731999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.731999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.753338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:55.754850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:55.830634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1078/2146] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:55.847058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:55.847058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:55.871973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:55.905287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f699-a3fa-4653-8189-8fbadd6ece2e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:55.909340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:55.909950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:55.910226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:55.910490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:55.920873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:55.922449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d9c68-cd17-4e10-ae3d-3957c4744a61 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:55.934571 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1069/2147] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:50:56.204488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.205940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:56.218976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18771-b57a-41d3-a289-55fcbbc5a4f2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.224757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.225698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.226546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.227074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] PATH_INFO: `/v3/projects/b94fc1b5379a42a7923ee23e5043ed9f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.228689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b94fc1b5379a42a7923ee23e5043ed9f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.230626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.263432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:56.287935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b94fc1b5379a42a7923ee23e5043ed9f', 'request_id': 'req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:56.296264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b94fc1b5379a42a7923ee23e5043ed9f', 'request_id': 'req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:56.301921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b94fc1b5379a42a7923ee23e5043ed9f', 'request_id': 'req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:56.312625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] The token cache is being invalidate because project b94fc1b5379a42a7923ee23e5043ed9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:50:56.312907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:56.313580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9cb23d00-e3a1-4724-8971-ef7a50a13eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:50:56.315332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1079/2148] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:50:56 2022] DELETE /identity/v3/projects/b94fc1b5379a42a7923ee23e5043ed9f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:56.321203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.321720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.322156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.343119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:56.344335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:56.415696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1070/2149] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:56.423648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.423932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.424207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.444408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:56.445998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:56.454994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1080/2150] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:56.459250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.469809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8229d1-1305-45da-b943-9e33bf3d9cfc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.475778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.476374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.476671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.476919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.477562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:50:56.477865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.479502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.492038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1071/2151] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:50:56.496732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.509840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58f7e5-65cc-4e4d-a8cd-e64a03a75d4e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.518201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.518837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.519111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.519373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.520010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.521936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.557097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1081/2152] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:50:56 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:56.561896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.582263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71fee-c678-4266-bcba-d7398efbe933 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.647266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.647528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.647736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.648058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.648738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.650760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.677496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1072/2153] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:50:56 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:50:56.681667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.693188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44065698-b3f7-437d-b7c5-7ad711c04c8b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.698867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.699567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.699879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.700184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.700906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.702668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.714003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1082/2154] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:56.717535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.728440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ffc0f-c71b-42a1-8d48-b5bc3028f61d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.733755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.734348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.734621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.734921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.735561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.737163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812461e-14fa-4b8f-999f-410a852d9a65 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.748039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1073/2155] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:50:56.752832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.763398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec061f-143e-4b87-88f9-1a00fefeea6b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:56.768572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:56.769222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.769496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.769763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] PATH_INFO: `/v3/projects/6bafae6685e248d2ac4fab672648e941/users/09f0647075204ba183fb1e5beb2c5930/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.785619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bafae6685e248d2ac4fab672648e941, user_id=09f0647075204ba183fb1e5beb2c5930, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:56.788352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:56.803267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1083/2156] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:50:56 2022] PUT /identity/v3/projects/6bafae6685e248d2ac4fab672648e941/users/09f0647075204ba183fb1e5beb2c5930/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:50:56.808315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165ddc-c491-481f-a425-aa5836973910 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:56.808595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165ddc-c491-481f-a425-aa5836973910 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:56.808866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165ddc-c491-481f-a425-aa5836973910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:56.827858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bc165ddc-c491-481f-a425-aa5836973910 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:56.829043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bc165ddc-c491-481f-a425-aa5836973910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:56.862663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:56.902025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1074/2157] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:50:56.912355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:56.914533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:57.003917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bb6a5-2ae7-43bb-b1c6-2b01ff0c2f3e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:57.008547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:57.009066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:57.009348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:57.009619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:57.020406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:57.021876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:57.032672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1084/2158] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:57.711458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb621-d6c0-48c2-a242-36ff80d5bb0d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:57.711744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb621-d6c0-48c2-a242-36ff80d5bb0d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:57.712038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb621-d6c0-48c2-a242-36ff80d5bb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:57.785134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b7eeb621-d6c0-48c2-a242-36ff80d5bb0d None None] MFA Rules not processed for user `09f0647075204ba183fb1e5beb2c5930`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:57.787610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b7eeb621-d6c0-48c2-a242-36ff80d5bb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:57.868647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1075/2159] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:50:57.881159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:50:57.882601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:57.951989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269978f7-3dc2-4a9e-83ff-83d7fd44697d service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:50:57.958408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:50:57.959284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:57.959575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:57.959936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:57.975235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:50:57.982473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:50:57.986465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfd615-cbee-4408-b1ef-33b0462d124f service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:50:57.997526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1085/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:50:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:50:59.562167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:50:59.562631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:50:59.562745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:50:59.583506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] MFA Rules not processed for user `09f0647075204ba183fb1e5beb2c5930`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:50:59.584890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:50:59.592490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1076/2161] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:05.149710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861e15-d98e-4d84-9ca7-6c203827b930 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:05.150195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861e15-d98e-4d84-9ca7-6c203827b930 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:05.150315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861e15-d98e-4d84-9ca7-6c203827b930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:05.170665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-46861e15-d98e-4d84-9ca7-6c203827b930 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:05.171911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-46861e15-d98e-4d84-9ca7-6c203827b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:05.248584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1086/2162] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:51:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:05.303333 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:05.305334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:05.424628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ed088-3f84-4ead-b1f0-a1b0ca93cdbf None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:05.436119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:05.437200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:05.437891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:05.438585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:05.452448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:05.455167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:05.467760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1077/2163] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:51:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:51:09.171139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9da30-c97c-42f2-88f4-9fa77157a726 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:09.172234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9da30-c97c-42f2-88f4-9fa77157a726 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:09.172747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9da30-c97c-42f2-88f4-9fa77157a726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:09.196983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cfb9da30-c97c-42f2-88f4-9fa77157a726 None None] MFA Rules not processed for user `ba00478a6ef74115b83726083026f574`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:09.202310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cfb9da30-c97c-42f2-88f4-9fa77157a726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:09.314605 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1087/2164] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:11.790087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:11.791919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:11.835060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa7ea8-d433-4dd0-88c0-e11d2ab3fbc1 service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:11.853436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:11.855686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:11.856159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:11.858162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] PATH_INFO: `/v3/users/1130ac61acfc4d6fa431258c857b17d8` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:11.862227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1130ac61acfc4d6fa431258c857b17d8)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:11.864261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:11.988840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:11.995934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1130ac61acfc4d6fa431258c857b17d8'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:11.997245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1130ac61acfc4d6fa431258c857b17d8'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:12.048449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1130ac61acfc4d6fa431258c857b17d8'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:12.066382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1130ac61acfc4d6fa431258c857b17d8'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:12.067540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1130ac61acfc4d6fa431258c857b17d8'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:12.128789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1078/2165] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:51:11 2022] DELETE /identity/v3/users/1130ac61acfc4d6fa431258c857b17d8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:12.131711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:12.132823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:12.133367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:12.179498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:12.181644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:12.299163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1088/2166] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:12.315720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:12.317941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:12.409994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623269-a5cf-4338-87c7-f900480db1aa admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:12.414285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:12.415036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:12.415822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:12.415822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:12.447192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:12.454070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a461eb-3efb-429b-a7cb-e24e8cb3db2b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:12.484089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1079/2167] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:13.634540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:13.637526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:13.657993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2b54-d365-4b21-bf5f-bd7f157c5884 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:13.665906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:13.666595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:13.666928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:13.667400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] PATH_INFO: `/v3/projects/797bba7315f942c987d4117271c6a13b` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:13.668576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=797bba7315f942c987d4117271c6a13b)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:13.670599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:13.759307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:13.782200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797bba7315f942c987d4117271c6a13b', 'request_id': 'req-27d3c882-0623-4952-a076-618d62401b9d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:13.802581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797bba7315f942c987d4117271c6a13b', 'request_id': 'req-27d3c882-0623-4952-a076-618d62401b9d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:13.819941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797bba7315f942c987d4117271c6a13b', 'request_id': 'req-27d3c882-0623-4952-a076-618d62401b9d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:13.852917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] The token cache is being invalidate because project 797bba7315f942c987d4117271c6a13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:51:13.858745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:13.866404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-27d3c882-0623-4952-a076-618d62401b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:13.872867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1089/2168] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:51:13 2022] DELETE /identity/v3/projects/797bba7315f942c987d4117271c6a13b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:14.960587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:14.960587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:14.960587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:14.987647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] MFA Rules not processed for user `1dde7fad075145c89f726ca85c6c00ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:14.988991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:15.074656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1080/2169] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:23.841119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:25.046743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=4,keystone:SELECT=64,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:30.868689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137c4a-29cc-4417-9712-201f178c4723 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:30.869268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137c4a-29cc-4417-9712-201f178c4723 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:30.869450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137c4a-29cc-4417-9712-201f178c4723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:30.892392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f8137c4a-29cc-4417-9712-201f178c4723 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:30.893608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f8137c4a-29cc-4417-9712-201f178c4723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:30.959211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1090/2170] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:30.971948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:30.973422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:31.044937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c00b-bef8-491c-9d29-0c6f77394d0f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:31.050379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:31.051092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:31.051369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:31.051721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:31.065445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:31.067841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd83f62-b784-4b10-ab1e-7cffdb88bb3d service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:31.080340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1081/2171] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:51:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:51:33.200576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700893-36f6-45ad-a327-d3101859724e None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.201131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700893-36f6-45ad-a327-d3101859724e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.201230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700893-36f6-45ad-a327-d3101859724e None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.202649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1091/2172] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:51:33.234314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832fb7-3c8b-4eec-b0ce-3ca89d3d4a86 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.234895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832fb7-3c8b-4eec-b0ce-3ca89d3d4a86 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.235014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832fb7-3c8b-4eec-b0ce-3ca89d3d4a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.271114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6c832fb7-3c8b-4eec-b0ce-3ca89d3d4a86 None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:33.272691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6c832fb7-3c8b-4eec-b0ce-3ca89d3d4a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.294657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700893-36f6-45ad-a327-d3101859724e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:33.296796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e1700893-36f6-45ad-a327-d3101859724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.369681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1082/2173] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:33.379390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180813-441c-4cd0-9880-9a23a7879658 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.380644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180813-441c-4cd0-9880-9a23a7879658 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.381046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180813-441c-4cd0-9880-9a23a7879658 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.383522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1083/2174] 10.222.0.25 () {54 vars in 961 bytes} [Sat Sep 3 03:51:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:51:33.386127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700893-36f6-45ad-a327-d3101859724e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:33.389445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.389941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.390441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.393545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:33.394276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.394578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.394940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] PATH_INFO: `/v3/users/658ef1a43f7d445a920e5c4f2b26e339` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.396514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=658ef1a43f7d445a920e5c4f2b26e339)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:33.398678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:33.415400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] MFA Rules not processed for user `15badc5a92d745b0896aed0f7c582e73`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:33.418083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.432361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1084/2175] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Sep 3 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:51:33.444986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:33.461507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e559bf-bb9e-4b7f-baa2-f38329ce5cab None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:33.461754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658ef1a43f7d445a920e5c4f2b26e339'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.469622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:33.470200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.470475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.470750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.481319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658ef1a43f7d445a920e5c4f2b26e339'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.481525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:33.483642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68026fd0-5a1a-4cdd-969f-12d92a0df753 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:33.492316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658ef1a43f7d445a920e5c4f2b26e339'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.493151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658ef1a43f7d445a920e5c4f2b26e339'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.494038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658ef1a43f7d445a920e5c4f2b26e339'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.500701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1085/2176] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:33.512174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1092/2177] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:51:33 2022] DELETE /identity/v3/users/658ef1a43f7d445a920e5c4f2b26e339 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:33.608964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.609368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.609516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.630250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:33.632066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.693891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1086/2178] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:33.707307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:33.712175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.726839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:33.728828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:33.746973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:33.757559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:33.802583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d0ef-aef4-4564-93e3-a109a1065e7c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:33.803440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b93b8-9db2-4499-8d1b-59c4599ae78e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:33.807682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:33.807955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:33.808428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.808611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.808717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.808890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.808993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.809161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] PATH_INFO: `/v3/projects/8dc12b2499eb4f8ab6f3e216b3f6ff3e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.810252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc12b2499eb4f8ab6f3e216b3f6ff3e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:33.812203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:33.817370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:33.818972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05215c99-b790-44dd-92be-690ff05a5594 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:33.827288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1093/2179] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:51:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:51:33.868312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc12b2499eb4f8ab6f3e216b3f6ff3e', 'request_id': 'req-424be67c-a50b-4759-b0e5-4caf5a440b11'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.878919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc12b2499eb4f8ab6f3e216b3f6ff3e', 'request_id': 'req-424be67c-a50b-4759-b0e5-4caf5a440b11'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.886050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc12b2499eb4f8ab6f3e216b3f6ff3e', 'request_id': 'req-424be67c-a50b-4759-b0e5-4caf5a440b11'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.899884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] The token cache is being invalidate because project 8dc12b2499eb4f8ab6f3e216b3f6ff3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:51:33.900410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.900869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-424be67c-a50b-4759-b0e5-4caf5a440b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:33.905694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1087/2180] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:51:33 2022] DELETE /identity/v3/projects/8dc12b2499eb4f8ab6f3e216b3f6ff3e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:33.907867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:33.908326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:33.908762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:33.930728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:33.931988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:34.018273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1094/2181] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:34.018462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.018462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.018683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.047727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:34.048940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:34.058623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1088/2182] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:34.063023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.074586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c19d-7c13-488c-9b84-c2235d8c49dd None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.083844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.084480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.084757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.085023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.085686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:51:34.085989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.090913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.109313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1095/2183] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:51:34.114070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.134175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114ae09-f869-41f9-8f72-c78918a30c03 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.141222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.141879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.142162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.142429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.143171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.145265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.186856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1089/2184] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:51:34 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:51:34.191018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.210846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f62e-3d87-40c6-be9a-e5ddf405fd76 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.309997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.310996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.311418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.311839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.312690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.314918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.354702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1096/2185] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:51:34 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:51:34.355721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.367905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17184a-9aa2-4328-9f8c-e2e8e9cf9bd1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.373881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.374616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.375171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.375695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.376569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.378408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.396640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1090/2186] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:51:34.402004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.418390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347055d-913b-4ec4-9eed-e55b4ebe8290 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.424589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.426227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.427184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.427958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.429991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.431987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36c1b-5ba5-414c-acf9-28d90bf9135b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.457005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.458604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1097/2187] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:51:34.471932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de941f7-fd4f-4637-9008-5a887a49efdd admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.477732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.478560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.478970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.479361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] PATH_INFO: `/v3/projects/a71aa6f3cafc46b78cf96f2b4aba05ca/users/b3c8ba9345354aa9878f5567cb1c6223/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.508507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71aa6f3cafc46b78cf96f2b4aba05ca, user_id=b3c8ba9345354aa9878f5567cb1c6223, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.511940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.533780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1091/2188] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:51:34 2022] PUT /identity/v3/projects/a71aa6f3cafc46b78cf96f2b4aba05ca/users/b3c8ba9345354aa9878f5567cb1c6223/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:34.550186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.550531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.550766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.587962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] MFA Rules not processed for user `b3c8ba9345354aa9878f5567cb1c6223`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:34.589310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:34.598124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:34.678872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1098/2189] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:34.691167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:34.692994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:34.749116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:34.771732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef94de-e75c-4bb2-aa1e-b3ba76ee35df admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:34.779527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:34.784341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:34.784672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:34.784957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:34.805952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:34.807690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9d58a-f579-4b75-8e40-2c50b9ddfb4f service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:34.830361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1092/2190] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:44.674997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=23 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:44.815222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:46.347622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:46.417687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea31d3-a3ed-43a3-be06-adb9b69a4295 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:46.423299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:46.423947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:46.424227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:46.424503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] PATH_INFO: `/v3/users/b3c8ba9345354aa9878f5567cb1c6223` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:46.425653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c8ba9345354aa9878f5567cb1c6223)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:46.427873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:46.483254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c8ba9345354aa9878f5567cb1c6223'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:46.496317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c8ba9345354aa9878f5567cb1c6223'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:46.503141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c8ba9345354aa9878f5567cb1c6223'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:46.503847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c8ba9345354aa9878f5567cb1c6223'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:46.504433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c8ba9345354aa9878f5567cb1c6223'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:46.519345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1099/2191] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:51:46 2022] DELETE /identity/v3/users/b3c8ba9345354aa9878f5567cb1c6223 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:46.523806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:46.524572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:46.524572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:46.542964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:46.544142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:46.611245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1093/2192] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:46.621923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:46.623441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:46.688158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fce9df-69be-49db-907e-ed55405d8a95 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:46.692470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:46.693054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:46.693364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:46.693636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:46.703176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:46.704613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ce33b-f97f-43ca-84bb-5f5044c19d4a service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:46.714212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1100/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:47.224228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.225929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:47.242536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e988c5-5c6c-4c0a-8c9c-4b457312e1f4 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.250441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.251141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.254964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.254964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] PATH_INFO: `/v3/projects/a71aa6f3cafc46b78cf96f2b4aba05ca` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.256718 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71aa6f3cafc46b78cf96f2b4aba05ca)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.258793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.309462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71aa6f3cafc46b78cf96f2b4aba05ca', 'request_id': 'req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:47.316861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71aa6f3cafc46b78cf96f2b4aba05ca', 'request_id': 'req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:47.322955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71aa6f3cafc46b78cf96f2b4aba05ca', 'request_id': 'req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:47.334593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] The token cache is being invalidate because project a71aa6f3cafc46b78cf96f2b4aba05ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:51:47.335033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:47.335762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f8e4d00a-7eeb-4b0c-8fe6-8cd8e314d48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:51:47.337385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1094/2194] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:51:47 2022] DELETE /identity/v3/projects/a71aa6f3cafc46b78cf96f2b4aba05ca => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:47.342385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.342691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.343004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.363664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:47.364880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:47.381784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:47.429692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1101/2195] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:47.437771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.438058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.438356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.458145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:47.459390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:47.467503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1095/2196] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:47.473043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.484370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bde747-2c6b-4319-a0a7-03a2152caae7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.490307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.493223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.493223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.493223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.493223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:51:47.493223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.493989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.506844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1102/2197] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:51:47.511862 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.522010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48078be-b9b1-44b1-b7f3-b5acaec03ada None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.529169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.529760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.530039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.530302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.530971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.532888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.570845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1096/2198] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:51:47 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:51:47.572869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.583203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95b233-c65c-46b5-94e2-a187efdb7c21 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.642579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.643330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.643607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.643899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.644529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.646463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.669111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1103/2199] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:51:47 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:51:47.673297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.684231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ad44-4f16-4b1f-bb74-68a9a2c16a22 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.690363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.691065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.691349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.691620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.692267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.694101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.705577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1097/2200] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:51:47.709679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.721047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b5d4-6709-4bbf-99b6-f16fb161d378 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.725896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.726499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.726773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.727069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.727708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.729295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07dcad-e863-48af-b357-7a0869ab6079 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.740629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1104/2201] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:51:47.745062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.755301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b15347-38c6-4642-a474-cba0a19b2e09 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.761222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.761906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.762166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.762440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] PATH_INFO: `/v3/projects/5ced06c15af249bf92f546d0a0bfba23/users/687629f84f1f408d8e80a5a1b28eb787/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.781767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ced06c15af249bf92f546d0a0bfba23, user_id=687629f84f1f408d8e80a5a1b28eb787, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.784549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.799809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1098/2202] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:51:47 2022] PUT /identity/v3/projects/5ced06c15af249bf92f546d0a0bfba23/users/687629f84f1f408d8e80a5a1b28eb787/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:47.805462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8e404-2361-4ed9-a19a-8f50b2826c51 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.805926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8e404-2361-4ed9-a19a-8f50b2826c51 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.806283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8e404-2361-4ed9-a19a-8f50b2826c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.825431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5ac8e404-2361-4ed9-a19a-8f50b2826c51 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:47.826558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5ac8e404-2361-4ed9-a19a-8f50b2826c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:47.885312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1105/2203] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:47.895496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:47.897360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:47.923553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:47.966029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cf1d-fc75-4e4a-8414-5232f1ecdceb admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:47.970162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:47.970826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:47.971094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:47.971355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:47.980689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:47.982462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:47.992424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1099/2204] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:52.367353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.367808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.368587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.423361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] MFA Rules not processed for user `687629f84f1f408d8e80a5a1b28eb787`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:52.424584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:52.441733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:52.542785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1106/2205] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:52.543377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.553775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec81f-4f7c-4627-88e6-b1b714c6953b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.559448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.560309 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.560735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.561197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.561868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.564013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.599274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1100/2206] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:51:52 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:51:52.603296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.615408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eb00a-c2d8-437d-a07e-ec0feb997847 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.687734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.688475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.688892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.689091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.689746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.692002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.718903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1107/2207] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:51:52 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:51:52.720618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.732980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775540e-8c4f-4500-ab87-6ba8df6faac0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.741790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.742422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.742697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.743006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.743758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.745377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.757335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1101/2208] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:51:52.761526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.774314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7968-a92a-4214-906b-e9578e1936cd admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.779228 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.779928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.780187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.780464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] PATH_INFO: `/v3/projects/d62871dac47946038b561e2467c53461/users/4b3d38cf375547558829cbe46b7a62e5/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.797076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62871dac47946038b561e2467c53461, user_id=4b3d38cf375547558829cbe46b7a62e5, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.799913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.817487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1108/2209] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:51:52 2022] PUT /identity/v3/projects/d62871dac47946038b561e2467c53461/users/4b3d38cf375547558829cbe46b7a62e5/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:51:52.821585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.832619 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758a86f-371e-4571-ae88-7862e32a978f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.921369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.921987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.922263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.922538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.923416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.925634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f69a82-4972-4158-8c4a-8b4c1aca941e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.942696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1102/2210] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:51:52.943973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:52.955035 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852dfbd-6b78-439a-be81-fad5ced50e40 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:52.964203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:52.964896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:52.965177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:52.965451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] PATH_INFO: `/v3/projects/d62871dac47946038b561e2467c53461/users/4b3d38cf375547558829cbe46b7a62e5/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:52.967492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62871dac47946038b561e2467c53461, user_id=4b3d38cf375547558829cbe46b7a62e5, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:52.970349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6a08-c8c2-4655-bdec-040770139838 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:52.984192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1109/2211] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:51:52 2022] PUT /identity/v3/projects/d62871dac47946038b561e2467c53461/users/4b3d38cf375547558829cbe46b7a62e5/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:51:53.020487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.020760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.021211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.022681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1103/2212] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:51:53.029837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620657-a233-40ad-8ca6-b68ecb205ea9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.030376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620657-a233-40ad-8ca6-b68ecb205ea9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.032210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620657-a233-40ad-8ca6-b68ecb205ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.061054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ee620657-a233-40ad-8ca6-b68ecb205ea9 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:53.062285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ee620657-a233-40ad-8ca6-b68ecb205ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:53.171104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1110/2213] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:53.183593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:53.185696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:53.255547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eeadf0-fcbe-4696-83da-d7895330ca50 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:53.260635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:53.261369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.261748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.262137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.276351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:53.278246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f682c7a-6042-41c5-9bbd-33b2694b06fd service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:53.292278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1104/2214] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:53.368927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.369489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.369612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.371301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1111/2215] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:51:53.376484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9c12-6d73-455a-abbf-178f4781fbdf None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.376855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9c12-6d73-455a-abbf-178f4781fbdf None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.377146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9c12-6d73-455a-abbf-178f4781fbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.408373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ce3f9c12-6d73-455a-abbf-178f4781fbdf None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:53.409682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ce3f9c12-6d73-455a-abbf-178f4781fbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:53.442094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:53.480095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1105/2216] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:51:53.495551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:53.498956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:53.511635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:51:53.522274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2ec70-b67b-41c7-bf68-bd7dc8772da0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:53.526288 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:53.527049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.527384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.527879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.536299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:53.537920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3186-8f7b-4ef5-bf62-95269b626dad service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:53.551004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1112/2217] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:53.645792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb467-4913-49c7-ae40-180477dd05a2 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.646241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb467-4913-49c7-ae40-180477dd05a2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.646362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb467-4913-49c7-ae40-180477dd05a2 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.647804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1106/2218] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:51:53.654604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda999-2e0c-40ea-9fe6-2247fe8d372a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.654991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda999-2e0c-40ea-9fe6-2247fe8d372a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.655264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda999-2e0c-40ea-9fe6-2247fe8d372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.676694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b9fda999-2e0c-40ea-9fe6-2247fe8d372a None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:53.677896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b9fda999-2e0c-40ea-9fe6-2247fe8d372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:53.691295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1113/2219] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:53.702898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bb467-4913-49c7-ae40-180477dd05a2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:53.716091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bb467-4913-49c7-ae40-180477dd05a2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:53.721212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:53.721864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:53.722232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:53.722457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:53.736863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:53.738316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:53.753388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1107/2220] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:57.964947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d0be-9ba6-4478-b1ed-1353a5b6b4d8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:57.965537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d0be-9ba6-4478-b1ed-1353a5b6b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:57.965537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d0be-9ba6-4478-b1ed-1353a5b6b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:57.995394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1fe6d0be-9ba6-4478-b1ed-1353a5b6b4d8 None None] MFA Rules not processed for user `4b3d38cf375547558829cbe46b7a62e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:57.996649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1fe6d0be-9ba6-4478-b1ed-1353a5b6b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:58.068823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1114/2221] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:51:58.082742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:51:58.086838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:58.102471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6030d-6778-4070-994d-c870b2e07e09 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:51:58.108678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102a348-3870-4096-8f71-493c1788094a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:51:58.109385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a348-3870-4096-8f71-493c1788094a service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:58.109733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a348-3870-4096-8f71-493c1788094a service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:58.109923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a348-3870-4096-8f71-493c1788094a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:58.173814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102a348-3870-4096-8f71-493c1788094a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:51:58.175511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102a348-3870-4096-8f71-493c1788094a service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:51:58.187828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1108/2222] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:51:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:51:59.290206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:51:59.291124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:51:59.291545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:51:59.315285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] MFA Rules not processed for user `687629f84f1f408d8e80a5a1b28eb787`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:51:59.316831 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:51:59.335888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1115/2223] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:52:01.062612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:01.063190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:01.063368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:01.084449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:52:01.085769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:01.151884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1109/2224] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:52:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:52:01.165760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:52:01.167493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:01.239869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59677e0-23f2-4aca-9336-c4dd7c4baf5a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:52:01.245747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:52:01.246979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:01.247588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:01.247770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:01.257398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:52:01.259091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6341f-4fdc-4d9a-b6cb-a2d22eecd5cf service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:52:01.268080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1116/2225] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:52:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:52:11.143616 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=69 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:52:11.264858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=77 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:52:16.476384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:52:16.477929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:16.561396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30c9c5-396f-46e6-a4aa-a8be4eb513e9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:52:16.565052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:52:16.565671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:16.565964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:16.566221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:16.568318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:16.581561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:52:16.583204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:52:16.595019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1110/2226] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:52:26.590918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=32 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:52:28.679280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b594-e1f2-42a7-83cf-6f0468e49657 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:28.679809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b594-e1f2-42a7-83cf-6f0468e49657 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:28.679809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b594-e1f2-42a7-83cf-6f0468e49657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:28.699168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8c08b594-e1f2-42a7-83cf-6f0468e49657 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:52:28.700358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8c08b594-e1f2-42a7-83cf-6f0468e49657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:28.708058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1117/2227] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:52:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:52:28.730209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:52:28.742734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b901e47-acf1-4f8f-a25b-c7fb7d784025 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:52:28.748065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:52:28.748661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:28.748941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:28.749207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:28.758284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:52:28.760106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:52:28.768646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1111/2228] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:52:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:52:31.418594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9025-2697-4a4b-ba36-e37f1b1460cc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:31.419145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9025-2697-4a4b-ba36-e37f1b1460cc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:31.419145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9025-2697-4a4b-ba36-e37f1b1460cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:31.439665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-059a9025-2697-4a4b-ba36-e37f1b1460cc None None] MFA Rules not processed for user `09f0647075204ba183fb1e5beb2c5930`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:52:31.441110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-059a9025-2697-4a4b-ba36-e37f1b1460cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:52:31.498887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1118/2229] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:52:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:52:38.765108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:52:40.275239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:52:40.285037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b980-db32-4571-88ff-1f79115cb9cc service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:52:40.289217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:52:40.289809 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:52:40.290108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:52:40.290377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:52:40.303921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:52:40.305921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a743eb-c0c8-4b54-be31-409383a9bef7 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:52:40.320539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1112/2230] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:52:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:52:41.491717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=28 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:52:50.313618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:53:11.038255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51669b9-21c9-49ac-950f-d74e75d35ea5 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:11.038789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51669b9-21c9-49ac-950f-d74e75d35ea5 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:11.038789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51669b9-21c9-49ac-950f-d74e75d35ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:11.060149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b51669b9-21c9-49ac-950f-d74e75d35ea5 None None] MFA Rules not processed for user `1dde7fad075145c89f726ca85c6c00ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:53:11.061393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b51669b9-21c9-49ac-950f-d74e75d35ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:11.124110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1119/2231] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:53:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:53:11.429166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ac46-2434-4444-841d-f48177b465bb None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:11.429661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ac46-2434-4444-841d-f48177b465bb None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:11.429661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ac46-2434-4444-841d-f48177b465bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:11.448467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9379ac46-2434-4444-841d-f48177b465bb None None] MFA Rules not processed for user `ba00478a6ef74115b83726083026f574`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:53:11.449668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9379ac46-2434-4444-841d-f48177b465bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:11.513659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1113/2232] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:53:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:53:21.117323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=25 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:53:21.500182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=25 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:53:32.515859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:32.516367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:32.516534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:32.535815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] MFA Rules not processed for user `31a8c62bd81f4c1f863bd952b6976276`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:53:32.537068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:32.596171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1120/2233] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:53:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:53:35.143209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379ac46-2434-4444-841d-f48177b465bb None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:53:35.145334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9379ac46-2434-4444-841d-f48177b465bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:35.218198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379ac46-2434-4444-841d-f48177b465bb None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:53:35.223327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:53:35.224225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:35.224659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:35.225094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] PATH_INFO: `/v3/users/09f0647075204ba183fb1e5beb2c5930` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:35.226379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f0647075204ba183fb1e5beb2c5930)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:53:35.228697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:53:35.300090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f0647075204ba183fb1e5beb2c5930'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.300774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f0647075204ba183fb1e5beb2c5930'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.315053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f0647075204ba183fb1e5beb2c5930'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.323870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f0647075204ba183fb1e5beb2c5930'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.327178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f0647075204ba183fb1e5beb2c5930'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.346018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1114/2234] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:53:35 2022] DELETE /identity/v3/users/09f0647075204ba183fb1e5beb2c5930 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:53:35.593763 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:53:35.612963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1722-c479-4383-ab3a-67a0fa76d247 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:53:35.697112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:53:35.698067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:35.698496 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:35.698962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] PATH_INFO: `/v3/projects/6bafae6685e248d2ac4fab672648e941` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:35.700233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bafae6685e248d2ac4fab672648e941)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:53:35.703022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:53:35.764820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bafae6685e248d2ac4fab672648e941', 'request_id': 'req-4540d135-b73a-466f-9fd7-127dd20f224b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.776299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bafae6685e248d2ac4fab672648e941', 'request_id': 'req-4540d135-b73a-466f-9fd7-127dd20f224b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.784518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bafae6685e248d2ac4fab672648e941', 'request_id': 'req-4540d135-b73a-466f-9fd7-127dd20f224b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.798083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] The token cache is being invalidate because project 6bafae6685e248d2ac4fab672648e941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:53:35.798545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.799569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:53:35.806903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1121/2235] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:53:35 2022] DELETE /identity/v3/projects/6bafae6685e248d2ac4fab672648e941 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:53:45.341671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:53:45.794264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:53:57.689787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:53:57.691336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:57.748572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e53d-6a78-47a5-8307-dbd28eb1d585 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:53:57.799795 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:53:57.800312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:57.800590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:57.800860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:57.802709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:57.874031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:53:57.875498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e571e-2ce8-4687-b4e3-2f623b83f6ab service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:53:57.886517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1115/2236] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:53:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:53:59.071480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:53:59.073143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:59.088964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540d135-b73a-466f-9fd7-127dd20f224b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:53:59.093931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:53:59.094491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:53:59.095497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:53:59.095497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:53:59.097174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:53:59.112656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:53:59.114095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:53:59.125603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1122/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:53:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:07.665968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:07.666475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:07.666611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:07.686181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] MFA Rules not processed for user `58361634962b4b7884d51be90b683b42`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:07.687423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:07.741818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1116/2238] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:09.120574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:14.129611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:14.131184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:14.144861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f54e-d54e-4105-bb93-cd456f788cd1 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:14.150942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:14.151547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:14.151823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:14.152090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:14.154020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:14.165477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:14.167227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:14.177950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1123/2239] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:15.299636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:15.301269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:15.312207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:15.319875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d923ba9-e862-4cc6-b5cc-b382dca1e636 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:15.326625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:15.327268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:15.327545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:15.327814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:15.329726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:15.391078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:15.394733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:15.404859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1117/2240] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:17.785075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:17.795086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a930c-0b93-4fe0-b327-c08bf4638fc9 service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:17.799214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:17.799813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:17.800123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:17.800384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:17.802379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:17.869773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:17.871417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:17.881626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1124/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:19.313030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:19.324628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c6c4f-18fa-4c4e-a266-5410dc9a99d3 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:19.330648 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:19.331385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:19.331657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:19.331933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:19.333827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:19.395366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:19.397019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:19.409376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1118/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:21.468151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:21.469767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:21.483201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85633e13-3a04-4821-9a84-42c8eeac6ab1 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:21.487821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:21.488594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:21.488874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:21.489144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] PATH_INFO: `/v3/users/ba00478a6ef74115b83726083026f574` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:21.490267 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba00478a6ef74115b83726083026f574)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:21.492210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:21.551197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba00478a6ef74115b83726083026f574'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.563729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba00478a6ef74115b83726083026f574'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.572055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba00478a6ef74115b83726083026f574'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.572788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba00478a6ef74115b83726083026f574'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.573681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c2e7888e-6c14-45e8-a0c4-cfe63a43a414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba00478a6ef74115b83726083026f574'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.588062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1125/2243] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:21 2022] DELETE /identity/v3/users/ba00478a6ef74115b83726083026f574 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:21.766632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:21.778909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d242a7-bb7e-46a2-b3c0-97f62d7161a1 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:21.852008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:21.852639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:21.852921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:21.853194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] PATH_INFO: `/v3/projects/f3077cfc588042e0a1becefe1363385c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:21.854296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3077cfc588042e0a1becefe1363385c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:21.856261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:21.903964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:21.915249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3077cfc588042e0a1becefe1363385c', 'request_id': 'req-700146de-883f-4fc9-bd2f-49d1f1df2444'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.924606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3077cfc588042e0a1becefe1363385c', 'request_id': 'req-700146de-883f-4fc9-bd2f-49d1f1df2444'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.930671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3077cfc588042e0a1becefe1363385c', 'request_id': 'req-700146de-883f-4fc9-bd2f-49d1f1df2444'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.947153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] The token cache is being invalidate because project f3077cfc588042e0a1becefe1363385c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:21.947440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.948211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-700146de-883f-4fc9-bd2f-49d1f1df2444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:21.949918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1119/2244] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:21 2022] DELETE /identity/v3/projects/f3077cfc588042e0a1becefe1363385c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:21.955413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:21.955680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:21.955985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:21.976492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:21.977684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:22.028743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:22.043336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1126/2245] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:22.051681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.051904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.052179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.073835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:22.075086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:22.083351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1120/2246] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:22.087702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.097464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0459fed-2ffc-43c8-aa06-dedc1f68fe39 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.102421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.103064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.103378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.103635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.104279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:54:22.104577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.106174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.122466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1127/2247] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:54:22.123694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.133034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0382-9362-4d86-baf7-e50cd3584aa2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.138168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.138762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.139069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.139338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.139966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.141879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.172815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1121/2248] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:22.177066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.188381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790082-c3bb-43e7-8071-b81305247643 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.251505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.252219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.252497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.252771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.253408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.255376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.279179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1128/2249] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/users => generated 279 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:22.283088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.297117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf31430-a8b5-4d8c-ab95-2bd6c84d25f1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.302679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.303401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.303744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.304076 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.304798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.306491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.318002 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1122/2250] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:22.322253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.333089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59735ed5-f31d-40c8-838f-73e65bc81608 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.338582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.339259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.339557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.339823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.340462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.342064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ade857-503c-4520-8e12-f17121fe3331 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.353335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1129/2251] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:22.357897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.367788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e18f-0b18-40b2-b7bd-783d9461bcc1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.372315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.372995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.373273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.373547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] PATH_INFO: `/v3/projects/901d9fc7a0684168b9fdb9339d11dc45/users/3cb664c2a3854c80804d6490bc82748b/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.390452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901d9fc7a0684168b9fdb9339d11dc45, user_id=3cb664c2a3854c80804d6490bc82748b, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.393244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.408982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1123/2252] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:22 2022] PUT /identity/v3/projects/901d9fc7a0684168b9fdb9339d11dc45/users/3cb664c2a3854c80804d6490bc82748b/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:22.424492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.424993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.425417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.454699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] MFA Rules not processed for user `3cb664c2a3854c80804d6490bc82748b`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:22.455996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:22.524580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1130/2253] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:22.529390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.601175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569abce7-b603-494a-b8f6-8c2a40309671 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.609698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.610535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.610860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.611215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.612021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.614162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.645146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1124/2254] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/projects => generated 357 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:22.646725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.658175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe9ef3-4e0a-4584-97f7-4f77d9d45a02 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.705195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:22.719497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.720112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.720386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.720685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.721790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.723761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.746077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1131/2255] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:22.750086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.761069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a7e6-0093-4812-8d3f-9346de63fd0e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.767030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.767826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.768148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.768420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.769062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.770681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.782180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1125/2256] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:54:22.786863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.797287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b33d0-dec9-4c60-bd6e-9b0fc5553f23 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.802041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.802742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.803047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.803312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] PATH_INFO: `/v3/projects/45d57bd8dd774166930174cc47de30d4/users/fab5bfc2941e43bf812d3cb64387e078/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.818581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d57bd8dd774166930174cc47de30d4, user_id=fab5bfc2941e43bf812d3cb64387e078, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.821306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.836592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1132/2257] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:22 2022] PUT /identity/v3/projects/45d57bd8dd774166930174cc47de30d4/users/fab5bfc2941e43bf812d3cb64387e078/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:22.840655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.851609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ffb0-2caf-45a2-abfb-861530d69bbe admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.890041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:22.911569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.912302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.912645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.912975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.913692 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.915454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c167d1-255b-4dee-a000-3592592437e0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.925900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1126/2258] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:22.930101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:22.940537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb11bc9-c408-4120-9bb6-1a4a41b3f8a9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:22.945310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:22.945961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.946235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.946505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] PATH_INFO: `/v3/projects/45d57bd8dd774166930174cc47de30d4/users/fab5bfc2941e43bf812d3cb64387e078/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:22.948401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d57bd8dd774166930174cc47de30d4, user_id=fab5bfc2941e43bf812d3cb64387e078, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:22.951064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:22.961521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1133/2259] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:22 2022] PUT /identity/v3/projects/45d57bd8dd774166930174cc47de30d4/users/fab5bfc2941e43bf812d3cb64387e078/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:22.971926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971562-9970-4793-8e95-70166077d283 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:22.972211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971562-9970-4793-8e95-70166077d283 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:22.972483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971562-9970-4793-8e95-70166077d283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:23.000760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-08971562-9970-4793-8e95-70166077d283 None None] MFA Rules not processed for user `fab5bfc2941e43bf812d3cb64387e078`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:23.002049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-08971562-9970-4793-8e95-70166077d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:23.069817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1127/2260] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:23.081291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:23.083268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:23.141664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d186f-0664-45ad-bbf5-2340bfa8bc4a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:23.145630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:23.146256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:23.146532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:23.146796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:23.156802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:23.158393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:23.169119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1134/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:23.259500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971562-9970-4793-8e95-70166077d283 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:23.270292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971562-9970-4793-8e95-70166077d283 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:23.274355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:23.275019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:23.275298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:23.275569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:23.335101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:23.336453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14244403-7a38-4d27-9da8-46d2e553be3c service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:23.345802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1128/2262] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:30.058659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:30.060725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:30.145915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b493756-dfb7-45ed-9458-bcabbb0a11e7 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:30.151398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:30.152027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:30.152482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:30.152757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:30.155120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:30.200527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:30.221107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:30.222778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:30.232845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1135/2263] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:33.341303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=73 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:36.176012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:36.176621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:36.176621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:36.197932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] MFA Rules not processed for user `1dde7fad075145c89f726ca85c6c00ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:36.199183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:36.260106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1129/2264] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:36.581731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:36.583355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:36.640604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8596-cf1f-4e9b-b998-31cec6c5f1f6 service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:36.644989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:36.645586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:36.645858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:36.646160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:36.647912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:36.657257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:36.658664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:36.666609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1136/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:36.994062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:36.995639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:37.076951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5bf85-9be4-4da0-bf7b-08aaec0ad0a1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:37.081882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:37.082494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:37.082770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:37.083072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:37.085044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:37.097185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:37.098777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32d6e-08b7-4690-9fb6-ba01e591238e service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:37.109568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1130/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:38.712582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:38.783275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb744-116d-49fb-ad46-1391ba09469a service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:38.789488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:38.790153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:38.790429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:38.790696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] PATH_INFO: `/v3/users/3cb664c2a3854c80804d6490bc82748b` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:38.791901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb664c2a3854c80804d6490bc82748b)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:38.793877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:38.845737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb664c2a3854c80804d6490bc82748b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:38.858405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb664c2a3854c80804d6490bc82748b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:38.866147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb664c2a3854c80804d6490bc82748b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:38.867015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb664c2a3854c80804d6490bc82748b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:38.867706 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb664c2a3854c80804d6490bc82748b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:38.885742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1137/2267] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:38 2022] DELETE /identity/v3/users/3cb664c2a3854c80804d6490bc82748b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:38.887748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:38.888238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:38.888365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:38.909815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:38.911182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:38.975334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1131/2268] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:38.985905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:38.987430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:38.992880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:39.050262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb07e6-ab09-4b8a-8d90-423e010b5448 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:39.055088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:39.055717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:39.055999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:39.056272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:39.066488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:39.068319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:39.078433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1138/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:39.370868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:39.374465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:39.391483 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37417b2-3a99-4ae5-8069-b2239395496b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:39.396475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:39.397186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:39.397459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:39.397773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] PATH_INFO: `/v3/projects/901d9fc7a0684168b9fdb9339d11dc45` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:39.399546 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901d9fc7a0684168b9fdb9339d11dc45)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:39.401555 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:39.442060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:39.460431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901d9fc7a0684168b9fdb9339d11dc45', 'request_id': 'req-584c0d7a-dd3f-4396-aab7-94085e8fe344'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.468498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901d9fc7a0684168b9fdb9339d11dc45', 'request_id': 'req-584c0d7a-dd3f-4396-aab7-94085e8fe344'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.482699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901d9fc7a0684168b9fdb9339d11dc45', 'request_id': 'req-584c0d7a-dd3f-4396-aab7-94085e8fe344'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.496948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] The token cache is being invalidate because project 901d9fc7a0684168b9fdb9339d11dc45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:39.497395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.497926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.499681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1132/2270] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:39 2022] DELETE /identity/v3/projects/901d9fc7a0684168b9fdb9339d11dc45 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:39.504213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:39.518680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8020d1-9472-4fdb-9be7-3daa3721c2eb service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:39.581370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:39.582163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:39.582536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:39.582936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] PATH_INFO: `/v3/users/fab5bfc2941e43bf812d3cb64387e078` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:39.584181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab5bfc2941e43bf812d3cb64387e078)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:39.586573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:39.637860 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab5bfc2941e43bf812d3cb64387e078'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.657183 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab5bfc2941e43bf812d3cb64387e078'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.668662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab5bfc2941e43bf812d3cb64387e078'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.669817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab5bfc2941e43bf812d3cb64387e078'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.670760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-e2e63798-6c18-4d91-b583-12ea4531a135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab5bfc2941e43bf812d3cb64387e078'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:39.687451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1139/2271] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:39 2022] DELETE /identity/v3/users/fab5bfc2941e43bf812d3cb64387e078 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:39.972684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:39.975601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:39.991197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0d7a-dd3f-4396-aab7-94085e8fe344 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.084528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.085170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.085672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.085876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] PATH_INFO: `/v3/projects/45d57bd8dd774166930174cc47de30d4` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.087042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d57bd8dd774166930174cc47de30d4)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.088982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.139006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d57bd8dd774166930174cc47de30d4', 'request_id': 'req-841b4c46-3e71-4678-87db-681562ac7419'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:40.147252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d57bd8dd774166930174cc47de30d4', 'request_id': 'req-841b4c46-3e71-4678-87db-681562ac7419'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:40.153364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d57bd8dd774166930174cc47de30d4', 'request_id': 'req-841b4c46-3e71-4678-87db-681562ac7419'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:40.165109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] The token cache is being invalidate because project 45d57bd8dd774166930174cc47de30d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:40.165109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:40.165406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-841b4c46-3e71-4678-87db-681562ac7419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:40.166954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1133/2272] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:39 2022] DELETE /identity/v3/projects/45d57bd8dd774166930174cc47de30d4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:40.173060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.173483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.173610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.194889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:40.196120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:40.215189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:40.262575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1140/2273] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:40.270532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.270834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.271113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.291465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:40.292669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:40.300976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1134/2274] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:40.305370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.315472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95364a-d7f2-44a5-b1b2-f440c69c560d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.320200 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.320818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.321086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.321349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.321988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:54:40.322280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.323912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.335731 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1141/2275] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:54:40.339944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.349594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c14a-0e9b-40c5-99c4-7c3e4b2fee0c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.354631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.355319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.355663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.355932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.356995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.359052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.387765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1135/2276] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:40.391846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.402737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b76ee-139e-4a1c-9069-bf98e1cc5511 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.465043 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.465663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.465943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.466218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.466895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.468827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.490277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1142/2277] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:54:40.494601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.504820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5635b-c8e1-48f7-94c7-966da97bc739 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.509614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.510236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.510523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.510825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.511519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.513153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.524232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1136/2278] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:54:40.528247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.539195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bf1a8-5485-441e-b0a4-674812e06d8e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.544909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.545581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.545919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.546272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.547013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.548657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56b1fd-f3f3-4913-a830-fec22930e522 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.559948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1143/2279] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:40.564266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.574833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8ea3-6232-43af-ab56-9b61c6680817 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.580574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.581231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.581511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.581780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] PATH_INFO: `/v3/projects/4ea7e145d70c479b9b2dffd266ac86ae/users/55ef658353b94db1bfa057ee22bdfc32/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.585981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:40.599957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea7e145d70c479b9b2dffd266ac86ae, user_id=55ef658353b94db1bfa057ee22bdfc32, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.603468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.618469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1137/2280] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:40 2022] PUT /identity/v3/projects/4ea7e145d70c479b9b2dffd266ac86ae/users/55ef658353b94db1bfa057ee22bdfc32/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:40.630160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.630443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.630715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.660939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] MFA Rules not processed for user `55ef658353b94db1bfa057ee22bdfc32`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:40.662205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:40.727443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1144/2281] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:54:40.732647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.797815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b48ab-bf04-4c7c-ba7e-4313211cc871 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.802478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.803130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.803440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.803709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.804377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.806315 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.833875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1138/2282] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:40.838376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.850662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba04d-73ae-4139-81ea-fc442d86265d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.869430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:40.916390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.917016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.917299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.917570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.918234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.920224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.941073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1145/2283] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:40 2022] POST /identity/v3/users => generated 307 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:40.945178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.955515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e0f1-b31a-4d66-8141-b5974933ee17 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.960317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.960918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.961192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.961461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:40.962102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:40.963748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:40.974191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1139/2284] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:40.978493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:40.988745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ec1a4-03de-44d5-9a88-8267f1a59fb0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:40.993507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:40.994150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:40.994426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:40.994695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] PATH_INFO: `/v3/projects/4dc15092e57346ed882755e2518dab87/users/8607b0b5cd1c4984b68422f32f577bdc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.009917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc15092e57346ed882755e2518dab87, user_id=8607b0b5cd1c4984b68422f32f577bdc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.012585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.027065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1146/2285] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:40 2022] PUT /identity/v3/projects/4dc15092e57346ed882755e2518dab87/users/8607b0b5cd1c4984b68422f32f577bdc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:41.031049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:41.041446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa13f5-7668-4c84-9a1b-ff68f7a4c645 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:41.099255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:41.099864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.100244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.100517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.101155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.102773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309512ad-e0c4-47f3-8ca2-e17b06918d18 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.113287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1140/2286] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:41.117839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:41.128935 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c69a3c-be82-4d8a-8bca-e789006181fd admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:41.135117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:41.135787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.136094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.136360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] PATH_INFO: `/v3/projects/4dc15092e57346ed882755e2518dab87/users/8607b0b5cd1c4984b68422f32f577bdc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.138574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc15092e57346ed882755e2518dab87, user_id=8607b0b5cd1c4984b68422f32f577bdc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.141282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.152084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1147/2287] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:41 2022] PUT /identity/v3/projects/4dc15092e57346ed882755e2518dab87/users/8607b0b5cd1c4984b68422f32f577bdc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:41.163086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.163444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.163807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.197794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] MFA Rules not processed for user `8607b0b5cd1c4984b68422f32f577bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:41.199047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:41.245826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:41.272874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1141/2288] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:41 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:41.277416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:41.279238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:41.284180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:41.286471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:41.343840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fd075-3bc6-4979-8a23-5c21fc74c6cc admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:41.346762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e061d8-97a5-4d97-95e1-30cc996f8ea1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:41.347785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:41.348391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.348675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.349061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.350847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:41.350987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:41.351464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.351740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.352008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.362405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.363606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.363900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.365103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.374653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1142/2289] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:41 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:54:41.374876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1148/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:41.460519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:41.471011 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27539-d69f-4958-a6ac-29ec380b4a8b service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:41.475492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:41.476360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:41.476735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:41.477094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:41.539501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:41.540975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2b445-a9a0-4c1b-8b18-e4e49e6eeccf service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:41.549993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1143/2291] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:41 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:45.724413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:45.814333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca1ae-462b-4934-92cd-681a621f89b4 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:45.818894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:45.819542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:45.819937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:45.820216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] PATH_INFO: `/v3/users/55ef658353b94db1bfa057ee22bdfc32` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:45.821690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ef658353b94db1bfa057ee22bdfc32)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:45.823814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:45.840670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:45.859580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:45.860100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:45.860100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:45.876920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ef658353b94db1bfa057ee22bdfc32'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:45.883287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:45.884524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:45.891298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ef658353b94db1bfa057ee22bdfc32'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:45.898668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ef658353b94db1bfa057ee22bdfc32'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:45.899474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ef658353b94db1bfa057ee22bdfc32'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:45.900093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-360388db-d901-4c3a-9316-956b15ee6d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ef658353b94db1bfa057ee22bdfc32'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:45.914708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1149/2292] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:45 2022] DELETE /identity/v3/users/55ef658353b94db1bfa057ee22bdfc32 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:45.919586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:45.919939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:45.920285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:45.940951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:45.942326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:45.946767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1144/2293] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:54:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:54:46.004431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:46.005306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1150/2294] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:54:46.005899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:46.014973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:46.016501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:46.042310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:46.090854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ea15-68d4-42be-ad44-2750e59a55ed None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:46.091340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49ad14-ce87-4819-9911-c4020422356a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:46.094727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:46.095336 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:46.095613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:46.095885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:46.106838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:46.108497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:46.125528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1151/2295] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:54:46.160713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:46.161336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:46.161742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:46.162019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:46.230659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:46.232748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:46.241767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1145/2296] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:54:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 03:54:46.470209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:46.471702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:46.485515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e0c98-7ac6-40c9-9423-3175dae3a8f3 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:46.491991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:46.492747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:46.493044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:46.493426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] PATH_INFO: `/v3/projects/4ea7e145d70c479b9b2dffd266ac86ae` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:46.497840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea7e145d70c479b9b2dffd266ac86ae)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:46.500100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:46.587597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:46.615979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea7e145d70c479b9b2dffd266ac86ae', 'request_id': 'req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.630926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea7e145d70c479b9b2dffd266ac86ae', 'request_id': 'req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.640903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea7e145d70c479b9b2dffd266ac86ae', 'request_id': 'req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.656311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] The token cache is being invalidate because project 4ea7e145d70c479b9b2dffd266ac86ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:46.656947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.658108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.660128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1152/2297] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:46 2022] DELETE /identity/v3/projects/4ea7e145d70c479b9b2dffd266ac86ae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:46.664501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:46.676719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f0d8-7a29-43f9-82fd-d034e5bfb3d0 service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:46.799452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:46.800252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:46.803193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:46.803489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] PATH_INFO: `/v3/users/8607b0b5cd1c4984b68422f32f577bdc` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:46.805233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8607b0b5cd1c4984b68422f32f577bdc)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:46.813694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:46.860570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:46.949689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8607b0b5cd1c4984b68422f32f577bdc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.950354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8607b0b5cd1c4984b68422f32f577bdc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.963444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8607b0b5cd1c4984b68422f32f577bdc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.970918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8607b0b5cd1c4984b68422f32f577bdc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:46.971671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-6e46a182-a451-4a89-9057-e3ad1fe42926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8607b0b5cd1c4984b68422f32f577bdc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.009018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1146/2298] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:46 2022] DELETE /identity/v3/users/8607b0b5cd1c4984b68422f32f577bdc => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:47.334526 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.337080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:47.353828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4197-98ea-49fd-a1b0-b711c21fd0b3 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.420389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.421492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.421969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.422415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] PATH_INFO: `/v3/projects/4dc15092e57346ed882755e2518dab87` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.423851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc15092e57346ed882755e2518dab87)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.426044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.488084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc15092e57346ed882755e2518dab87', 'request_id': 'req-b81d910f-08e2-4b14-a394-b09c3a5906a1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.497740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc15092e57346ed882755e2518dab87', 'request_id': 'req-b81d910f-08e2-4b14-a394-b09c3a5906a1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.506728 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc15092e57346ed882755e2518dab87', 'request_id': 'req-b81d910f-08e2-4b14-a394-b09c3a5906a1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.520924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] The token cache is being invalidate because project 4dc15092e57346ed882755e2518dab87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:47.521229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.522013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b81d910f-08e2-4b14-a394-b09c3a5906a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:47.523927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1153/2299] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:47 2022] DELETE /identity/v3/projects/4dc15092e57346ed882755e2518dab87 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:47.528931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.529218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.529480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.552820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:47.554108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:47.621068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1147/2300] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:54:47.630513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.630791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.631128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.652371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:47.653653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:47.662704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1154/2301] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:47.667243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.678321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959dfe4-a3d0-42d5-82ac-6e8f8f9061fd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.683056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.683658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.683937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.684224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.684872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:54:47.685173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.686854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.701166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1148/2302] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:54:47.705653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.720654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5decc-246f-4929-a561-c7f3fb15b2f8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.726216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.726878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.727162 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.727437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.728089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.730854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.766870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1155/2303] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:47 2022] POST /identity/v3/projects => generated 407 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:54:47.770375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.781348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee562-ad3e-4989-8588-a4b5e08720f3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.847230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.848358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.850126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.851112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.852949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.855325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.888653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1149/2304] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:47 2022] POST /identity/v3/users => generated 304 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:54:47.890026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.919322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa28e7-1072-493e-871e-d4168fb3b3c3 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.937719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.938725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.940226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.940655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.941375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.943359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.957540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1156/2305] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:47.963092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:47.976819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2b49f-02c7-44ef-b655-600accda3a75 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:47.988302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:47.988968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:47.989248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:47.989523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:47.990182 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:47.991990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88400d-721f-4678-9706-59e0b42bbd55 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:47.993590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:48.017392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1150/2306] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:48.018272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.034477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2a48-d6bb-436b-8a44-314fd630235d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.041069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.041892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.042208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.042560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] PATH_INFO: `/v3/projects/310cc64ef0924d18b78b4f2388e137f5/users/7768dd5b56d0421a9c9f5601f8b5c12d/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.052687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:48.070770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=310cc64ef0924d18b78b4f2388e137f5, user_id=7768dd5b56d0421a9c9f5601f8b5c12d, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.073824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.099151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1157/2307] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:48 2022] PUT /identity/v3/projects/310cc64ef0924d18b78b4f2388e137f5/users/7768dd5b56d0421a9c9f5601f8b5c12d/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:48.110080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.110605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.110605 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.167112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] MFA Rules not processed for user `7768dd5b56d0421a9c9f5601f8b5c12d`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:48.168929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:48.244643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1151/2308] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:48 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:48.249549 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.346331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff69ed-da3b-46dc-b2af-110c7f2e7a8e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.351278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.351928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.352206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.352478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.353125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.355041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.392463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1158/2309] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:48 2022] POST /identity/v3/projects => generated 409 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:54:48.394023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.406297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c37d-5667-4ac3-b8b1-82251306ba29 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.473658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.474493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.474884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.475227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.476015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.478031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.506921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.509014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1152/2310] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:48 2022] POST /identity/v3/users => generated 311 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:54:48.518511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b36272-a937-4783-bb8f-0a002e91d889 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.524725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.527296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.527720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.528676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.529648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.532164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.548709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1159/2311] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:48.552927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.564208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86918b-4fb7-40e2-9545-168f7efda478 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.569804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.570872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.571133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.571369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] PATH_INFO: `/v3/projects/6aac2973ccfb4b339a76044e0e4371d5/users/f916f0c8b1bd4f79840e7cb0d8944a77/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.592373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aac2973ccfb4b339a76044e0e4371d5, user_id=f916f0c8b1bd4f79840e7cb0d8944a77, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.595422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.619073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1153/2312] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:48 2022] PUT /identity/v3/projects/6aac2973ccfb4b339a76044e0e4371d5/users/f916f0c8b1bd4f79840e7cb0d8944a77/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:48.624195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.636321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f0a3d-eb7b-4225-91b0-bb1b4d2a3297 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.713014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.713697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.713977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.714251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.714934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.716736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484ea0-0a5d-4396-b066-a719a91dd3b5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.732705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.743155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1160/2313] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:54:48.744890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721823c-de55-4b2c-8fcf-aa17147d8fee admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:48.752224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:48.752879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.753152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.753430 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] PATH_INFO: `/v3/projects/6aac2973ccfb4b339a76044e0e4371d5/users/f916f0c8b1bd4f79840e7cb0d8944a77/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.755479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aac2973ccfb4b339a76044e0e4371d5, user_id=f916f0c8b1bd4f79840e7cb0d8944a77, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:48.758654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:48.772734 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1154/2314] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:54:48 2022] PUT /identity/v3/projects/6aac2973ccfb4b339a76044e0e4371d5/users/f916f0c8b1bd4f79840e7cb0d8944a77/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:48.783180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:48.784011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:48.784011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:48.816909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] MFA Rules not processed for user `f916f0c8b1bd4f79840e7cb0d8944a77`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:48.818092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:48.870887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.872766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:48.882356 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:48.915169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1161/2315] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:48 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:48.926022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:48.927724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:48.954202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:48.996014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f759-e3b3-4757-abe3-2b45f1302243 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:49.006644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbf2c8-f763-4c4a-b771-4bef10f42b97 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:49.007000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:49.007799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:49.008148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:49.011180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:49.013864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:49.014445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:49.015709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:49.016170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:49.016692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:49.028175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:49.038016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:49.041092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:49.044373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:49.051261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1162/2316] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:48 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:54:49.064231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1155/2317] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:49.338610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:49.358365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba0a1f-9a0d-4412-94ab-531c30a63a9a service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:49.362876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:49.363612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:49.363962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:49.364589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:49.501982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:49.504995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:49.522138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1163/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:49 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:54:50.476040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:50.492935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9d08b-d025-480a-9648-cc94622eaa33 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:50.499323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:50.499511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:50.499735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:50.500007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:50.502015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:50.514597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:50.516155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308a6cc-0808-42b7-a82f-09758acb8aff service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:50.537080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1156/2319] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:58.238069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:58.302451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a752d-decb-4804-bf97-e86103bce2fc service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:58.308205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:58.308859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:58.309132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:58.309396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] PATH_INFO: `/v3/users/7768dd5b56d0421a9c9f5601f8b5c12d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:58.310710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7768dd5b56d0421a9c9f5601f8b5c12d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:58.312754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:58.341927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:58.363910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7768dd5b56d0421a9c9f5601f8b5c12d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.376099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7768dd5b56d0421a9c9f5601f8b5c12d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.383026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7768dd5b56d0421a9c9f5601f8b5c12d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.383711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7768dd5b56d0421a9c9f5601f8b5c12d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.384310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7768dd5b56d0421a9c9f5601f8b5c12d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.397084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1164/2320] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:58 2022] DELETE /identity/v3/users/7768dd5b56d0421a9c9f5601f8b5c12d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:58.401609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:58.402128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:58.402128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:58.421768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:58.423114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:58.486976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1157/2321] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:58.498610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:58.500151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:58.559379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10652e-ccc4-41af-84a5-750c66ea736f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:58.563407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:58.564026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:58.564298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:58.564543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:58.574780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:58.576301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:58.588350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1165/2322] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:54:58.880088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:58.881601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:58.895321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5754ac-9a2d-44f1-ab15-97e0701d4c2a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:58.907954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:58.908629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:58.908904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:58.909172 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] PATH_INFO: `/v3/projects/310cc64ef0924d18b78b4f2388e137f5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:58.911288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=310cc64ef0924d18b78b4f2388e137f5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:58.912859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:58.989830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310cc64ef0924d18b78b4f2388e137f5', 'request_id': 'req-8774b614-00b1-4626-a4ba-97be3be31bdd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:58.999559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310cc64ef0924d18b78b4f2388e137f5', 'request_id': 'req-8774b614-00b1-4626-a4ba-97be3be31bdd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.006173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310cc64ef0924d18b78b4f2388e137f5', 'request_id': 'req-8774b614-00b1-4626-a4ba-97be3be31bdd'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.025296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] The token cache is being invalidate because project 310cc64ef0924d18b78b4f2388e137f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:59.025714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.026367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.028297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1158/2323] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:58 2022] DELETE /identity/v3/projects/310cc64ef0924d18b78b4f2388e137f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:59.032355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.042872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebfe53-c7d5-4c4e-847e-8bb74e8a5743 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:59.111783 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:59.112431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.112701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.112969 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] PATH_INFO: `/v3/users/f916f0c8b1bd4f79840e7cb0d8944a77` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.114114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f916f0c8b1bd4f79840e7cb0d8944a77)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:59.116084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:59.162018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f916f0c8b1bd4f79840e7cb0d8944a77'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.163129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:59.177085 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f916f0c8b1bd4f79840e7cb0d8944a77'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.184260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f916f0c8b1bd4f79840e7cb0d8944a77'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.185022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f916f0c8b1bd4f79840e7cb0d8944a77'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.185927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b52bba0e-c069-4e8b-bbd1-f45a0ddbcb93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f916f0c8b1bd4f79840e7cb0d8944a77'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.200420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1166/2324] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:54:59 2022] DELETE /identity/v3/users/f916f0c8b1bd4f79840e7cb0d8944a77 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:54:59.501964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.503763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:59.525088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b614-00b1-4626-a4ba-97be3be31bdd admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:59.557199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:54:59.587272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:59.587919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.588199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.588471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] PATH_INFO: `/v3/projects/6aac2973ccfb4b339a76044e0e4371d5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.589591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aac2973ccfb4b339a76044e0e4371d5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:59.591608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:59.641064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aac2973ccfb4b339a76044e0e4371d5', 'request_id': 'req-505a8dfe-ef7b-4132-965a-da16ef085384'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.649426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aac2973ccfb4b339a76044e0e4371d5', 'request_id': 'req-505a8dfe-ef7b-4132-965a-da16ef085384'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.655851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aac2973ccfb4b339a76044e0e4371d5', 'request_id': 'req-505a8dfe-ef7b-4132-965a-da16ef085384'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.667872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] The token cache is being invalidate because project 6aac2973ccfb4b339a76044e0e4371d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:54:59.668161 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.668858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-505a8dfe-ef7b-4132-965a-da16ef085384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:54:59.670393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1159/2325] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:54:59 2022] DELETE /identity/v3/projects/6aac2973ccfb4b339a76044e0e4371d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:54:59.675505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.676037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.676157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.697306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:59.698521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:59.761319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1167/2326] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:59.769062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.769366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.769643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.790864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:54:59.792198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:59.800381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1160/2327] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:54:59.804884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.816609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c648775-7449-4f2d-a2c3-9a7e4b34366a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:59.821943 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:59.822563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.823021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.823302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.823962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:54:59.824275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:59.825921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:59.839209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1168/2328] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:54:59.843744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.853487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9159c-19a6-4b07-9a42-745575b8004e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:59.858572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:59.859231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.859515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.859788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.860465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:59.862478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:59.894151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1161/2329] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:54:59 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:54:59.898527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.908812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdca-d187-4689-a2e8-0103c90fd761 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:54:59.947540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:54:59.950165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:54:59.971047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:54:59.971663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:54:59.971941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:54:59.972211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:54:59.972851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:54:59.974795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:54:59.997151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1169/2330] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:54:59 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:55:00.001401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.007985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03fb5-ea4b-44ab-b100-62a63fe07769 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.010827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8c05-f196-48f1-971a-671438e001f2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.011589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.012225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.012499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.012766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.014479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.015658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.016240 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.016571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.016788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.017426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.019027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.023788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.025202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.029058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1170/2331] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:55:00.032854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.033306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1162/2332] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:54:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:55:00.046483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614566c9-838f-47a9-a11c-410aab487a6c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.056062 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.056841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.057220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.057608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.059126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.060910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205721e-643e-4b6c-a732-3c05932afac4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.073574 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1171/2333] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:55:00.077739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.090831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a194cb-6d64-4dc4-ae0d-144f1aacc89b service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.096013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.096680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.096957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.097227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] PATH_INFO: `/v3/projects/a02f23c99aa9447fbd358f362deb4b70/users/486b92050727430bbb9870e63029a84b/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.114398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02f23c99aa9447fbd358f362deb4b70, user_id=486b92050727430bbb9870e63029a84b, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.117859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925078b4-727d-4f5a-a353-0fbb5bb6ecff admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.144790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1163/2334] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:55:00 2022] PUT /identity/v3/projects/a02f23c99aa9447fbd358f362deb4b70/users/486b92050727430bbb9870e63029a84b/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:00.146685 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.147160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.147435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.180229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d45a-7eba-46ea-8c42-c06f4c489f84 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.180514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d45a-7eba-46ea-8c42-c06f4c489f84 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.180783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d45a-7eba-46ea-8c42-c06f4c489f84 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.182296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1164/2335] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:55:00.185475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] MFA Rules not processed for user `486b92050727430bbb9870e63029a84b`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:00.186734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.191291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.191572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.191836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.198780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:00.217122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:00.218329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.228880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:00.287308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1172/2336] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:00.296235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.298003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.309742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1165/2337] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:00.324383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.326835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.376626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13795-d898-428b-a5e8-e08845a5eb78 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.380333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.380947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.381225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.381495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.389958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.391445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dc6a6-9231-44d6-87b8-56398ac0a357 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.400461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1173/2338] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:00.406438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224050c9-6857-4158-9d78-46aa039de4dd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.410385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.411015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.411291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.411558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.421986 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.423476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899e81d-3afa-460e-aa4c-86988a49f461 service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.434058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1166/2339] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:00.519564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.520031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.520577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] PATH_INFO: `/` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.522733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1174/2340] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 03:55:00.528584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a228-f682-4227-bee3-3703c7d23836 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.529305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a228-f682-4227-bee3-3703c7d23836 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.530116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a228-f682-4227-bee3-3703c7d23836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.564529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-18f4a228-f682-4227-bee3-3703c7d23836 None None] MFA Rules not processed for user `e5c0ba8119d54212a8c142353d27cd91`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:00.566157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-18f4a228-f682-4227-bee3-3703c7d23836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.658630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1167/2341] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:00.675695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.677537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.691782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa4ee0-bb5d-4ed2-93a8-562824fcd872 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.695900 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.696543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.696817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.697109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.705351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.706820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b1dd-8161-4120-bd64-1eaa30f5f9e4 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.717086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1175/2342] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:00.807769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb289d-e6ca-4df6-a7d9-e7fd5bb7fe20 None None] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.808099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb289d-e6ca-4df6-a7d9-e7fd5bb7fe20 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.808316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb289d-e6ca-4df6-a7d9-e7fd5bb7fe20 None None] PATH_INFO: `/` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.809684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1168/2343] 10.222.0.25 () {54 vars in 921 bytes} [Sat Sep 3 03:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 03:55:00.814781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b43b1-7087-4b01-8262-b3398b97d220 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.815161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b43b1-7087-4b01-8262-b3398b97d220 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.815520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b43b1-7087-4b01-8262-b3398b97d220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.835298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-289b43b1-7087-4b01-8262-b3398b97d220 None None] MFA Rules not processed for user `8e0e6be0fc61491fa25193834b4fa8af`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:00.836611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-289b43b1-7087-4b01-8262-b3398b97d220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:00.845887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1176/2344] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Sep 3 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:55:00.857114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb289d-e6ca-4df6-a7d9-e7fd5bb7fe20 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:00.868181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb289d-e6ca-4df6-a7d9-e7fd5bb7fe20 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:00.872156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:00.872774 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:00.873049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:00.873320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:00.883506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:00.884870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:00.895867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1169/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:05.271480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:05.271940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:05.272132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:05.292541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:05.293751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:05.352320 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1177/2346] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:55:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:05.364647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:05.366213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:05.412743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:05.435763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5c0b2-e642-40d3-8d91-455a91611982 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:05.441162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:05.441778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:05.442049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:05.442318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:05.450308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:05.451935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ada5d-9ac1-4ca2-b039-27e6f25ca828 service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:05.459699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1170/2347] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:55:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:55:15.345756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=83 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:15.456844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=12 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:20.523896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:20.589303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c54da-10e7-410e-88ee-6a0388ab5c9b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:20.594024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:20.594656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:20.594956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:20.595223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] PATH_INFO: `/v3/users/486b92050727430bbb9870e63029a84b` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:20.596952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=486b92050727430bbb9870e63029a84b)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:20.598979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:20.648465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486b92050727430bbb9870e63029a84b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:20.661606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486b92050727430bbb9870e63029a84b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:20.671525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486b92050727430bbb9870e63029a84b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:20.672463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486b92050727430bbb9870e63029a84b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:20.673548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486b92050727430bbb9870e63029a84b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:20.693648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1178/2348] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:55:20 2022] DELETE /identity/v3/users/486b92050727430bbb9870e63029a84b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:20.699113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70456-3a98-4e67-946b-071bd072097b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:20.699699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70456-3a98-4e67-946b-071bd072097b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:20.699839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70456-3a98-4e67-946b-071bd072097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:20.729158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2db70456-3a98-4e67-946b-071bd072097b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:20.730368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2db70456-3a98-4e67-946b-071bd072097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:20.825350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1171/2349] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:20.842514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:20.844090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:20.950902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0805c-b7ab-467b-90e3-b159ec7b5807 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:20.968996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:20.969916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:20.970782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:20.970782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:20.992225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:20.993913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308abe59-4267-4877-8de6-6d68ca359757 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.009138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1179/2350] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:21.446243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db70456-3a98-4e67-946b-071bd072097b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.447832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2db70456-3a98-4e67-946b-071bd072097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:21.461204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db70456-3a98-4e67-946b-071bd072097b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.466357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.467175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.467515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.467865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] PATH_INFO: `/v3/projects/a02f23c99aa9447fbd358f362deb4b70` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.469224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02f23c99aa9447fbd358f362deb4b70)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.471250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.529235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02f23c99aa9447fbd358f362deb4b70', 'request_id': 'req-3f47f453-07f8-4407-9f8a-99b158c0bd78'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:21.540249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02f23c99aa9447fbd358f362deb4b70', 'request_id': 'req-3f47f453-07f8-4407-9f8a-99b158c0bd78'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:21.546284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02f23c99aa9447fbd358f362deb4b70', 'request_id': 'req-3f47f453-07f8-4407-9f8a-99b158c0bd78'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:21.557548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] The token cache is being invalidate because project a02f23c99aa9447fbd358f362deb4b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:55:21.557830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:21.558651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3f47f453-07f8-4407-9f8a-99b158c0bd78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:21.560222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1172/2351] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:55:21 2022] DELETE /identity/v3/projects/a02f23c99aa9447fbd358f362deb4b70 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:21.565576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.566044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.566137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.586671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:21.587951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:21.607481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:21.654332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1180/2352] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:21.663562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.663846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.664122 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.684753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:21.685954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:21.695009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1173/2353] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:21.699542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.709842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e1deb-3ec9-4212-aa59-8dad069ac0c6 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.715674 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.716303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.716585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.716851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.717499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:55:21.717806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.719480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.731604 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1181/2354] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:55:21.736353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.748967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7185b4-3e69-46b8-84b7-d6c68310c64a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.755151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.755801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.756098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.756371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.757018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.758987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.795422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1174/2355] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:55:21 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:55:21.796765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.807617 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a244dc-07ae-47dd-aff7-ecce32ae4bd5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.873341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.874081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.874493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.874764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.875486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.877490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.906435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1182/2356] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:55:21 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:55:21.912579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.928267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493440f-e6d1-4dca-861b-d9bfe88b5951 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.934199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.934828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.935073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.935374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.936020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.937621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.947934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1175/2357] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:55:21.951882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.962303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218086-ba29-423b-b5fc-a95f94212236 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:21.967366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:21.967964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:21.968241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:21.968509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:21.969235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:21.971420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca9c7-aad7-451f-9931-3b567c04404f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:21.983655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1183/2358] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:55:21.988097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:21.998560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b6d74-7710-419f-8b5b-226d0e7c3b7e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:22.003636 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:22.004298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:22.004575 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:22.004961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] PATH_INFO: `/v3/projects/7ab2b09cab1a4218bb4387815334ef10/users/f1ff0c02ded7455a9f750e679c757398/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:22.020754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab2b09cab1a4218bb4387815334ef10, user_id=f1ff0c02ded7455a9f750e679c757398, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:22.023489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:22.039873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1176/2359] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:55:21 2022] PUT /identity/v3/projects/7ab2b09cab1a4218bb4387815334ef10/users/f1ff0c02ded7455a9f750e679c757398/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:22.050469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091710-e408-4d4a-bc94-7e81c986d939 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:22.050754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091710-e408-4d4a-bc94-7e81c986d939 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:22.051095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091710-e408-4d4a-bc94-7e81c986d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:22.079008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-88091710-e408-4d4a-bc94-7e81c986d939 None None] MFA Rules not processed for user `f1ff0c02ded7455a9f750e679c757398`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:22.080252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-88091710-e408-4d4a-bc94-7e81c986d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:22.146293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1184/2360] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:22.158253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:22.160250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:22.183285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:22.222720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac526b6-0793-4328-b0f6-3882aa734842 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:22.226720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:22.227399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:22.227642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:22.227941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:22.246276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:22.247808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:22.266857 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1177/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:24.423564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:24.424484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:24.424761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:24.447959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:24.449510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:24.464705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:24.523927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1185/2362] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:55:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:55:24.538418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:24.540725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:24.618792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b21d6-fe57-4bab-97a1-e080496942e4 service glance] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:24.624485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:24.625195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:24.625531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:24.625919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:24.639674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:24.642613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0ae0e-bb96-4043-888e-2b99e787432c service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:24.657849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1178/2363] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:55:34.516211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=17 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:34.652023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=49 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:48.269424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:48.343012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c1006-4689-4849-a082-81d845ff2d30 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:48.348067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:48.348704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:48.348984 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:48.349256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] PATH_INFO: `/v3/users/f1ff0c02ded7455a9f750e679c757398` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:48.350389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ff0c02ded7455a9f750e679c757398)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:48.352371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:48.404729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ff0c02ded7455a9f750e679c757398'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:48.416518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ff0c02ded7455a9f750e679c757398'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:48.423297 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ff0c02ded7455a9f750e679c757398'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:48.423961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ff0c02ded7455a9f750e679c757398'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:48.424582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ff0c02ded7455a9f750e679c757398'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:48.440117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1186/2364] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:55:48 2022] DELETE /identity/v3/users/f1ff0c02ded7455a9f750e679c757398 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:48.446645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:48.447251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:48.447251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:48.466695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:48.467948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:48.539073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1179/2365] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:48.549113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:48.550852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:48.611899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931bd0-6176-403c-a01e-fffaf12b990e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:48.616147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:48.616786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:48.617056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:48.617328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:48.626639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:48.628122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9635c34-735f-4fa5-a760-fb82e597f4f5 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:48.637629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1187/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:48.929038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:48.930574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:48.949890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2b40-1f07-4ac6-90a3-7b3ad7fbc7cc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:48.956888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:48.957876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:48.958230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:48.958561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] PATH_INFO: `/v3/projects/7ab2b09cab1a4218bb4387815334ef10` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:48.960056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab2b09cab1a4218bb4387815334ef10)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:48.962060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.023521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab2b09cab1a4218bb4387815334ef10', 'request_id': 'req-5f55e517-6d99-4230-8fc0-67bd1df39258'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:49.035400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab2b09cab1a4218bb4387815334ef10', 'request_id': 'req-5f55e517-6d99-4230-8fc0-67bd1df39258'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:49.041120 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab2b09cab1a4218bb4387815334ef10', 'request_id': 'req-5f55e517-6d99-4230-8fc0-67bd1df39258'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:49.052537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] The token cache is being invalidate because project 7ab2b09cab1a4218bb4387815334ef10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:55:49.052828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:49.053516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f55e517-6d99-4230-8fc0-67bd1df39258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:55:49.055100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1180/2367] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:55:48 2022] DELETE /identity/v3/projects/7ab2b09cab1a4218bb4387815334ef10 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:49.060221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.060503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.060778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.080833 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:49.082093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:49.099895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:49.145366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1188/2368] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:49.154403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.154686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.154981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.175656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:49.176985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:49.185554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1181/2369] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:49.189964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.200224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424b30-ca1e-4d82-b904-dc54573d35ce None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.206009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.206612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.206918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.207185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.207918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:55:49.208221 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.209810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.221857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1189/2370] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:55:49.226478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.236716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2647db-9489-46cf-897f-d978fc1656a3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.242780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.243559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.243844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.244115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.244768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.246749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.277922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1182/2371] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:55:49 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:55:49.282343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.293076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5835e-b4b6-43c6-9465-b063635aa3f8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.357874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.358489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.358769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.359077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.359726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.361644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.388250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1190/2372] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:55:49 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:55:49.392602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.405021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f699c-b47b-45b1-8d8d-93cc23bb7f38 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.412890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.413500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.413772 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.414044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.414690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.416343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.426491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1183/2373] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:55:49.430454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.440839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161194-1758-4996-8218-f73af6a36a3b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.446652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.447317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.447594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.447859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.448497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.450097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82e463-28a9-45d5-98e6-9eddebb365df admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.460410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1191/2374] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:55:49.464582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.474938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25de7-45fe-4c01-874f-ec93fcf6c467 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.480168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.481167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.481440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.481696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] PATH_INFO: `/v3/projects/78f83c7b81a54c9a936f140ca94e3d34/users/5594b2c1a2fd4bf8ad2f1c5d2c05b777/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.497967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f83c7b81a54c9a936f140ca94e3d34, user_id=5594b2c1a2fd4bf8ad2f1c5d2c05b777, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.500637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.516374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1184/2375] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:55:49 2022] PUT /identity/v3/projects/78f83c7b81a54c9a936f140ca94e3d34/users/5594b2c1a2fd4bf8ad2f1c5d2c05b777/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:55:49.589204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105993-d8b4-409e-82b2-d0996ffbfe77 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.590020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105993-d8b4-409e-82b2-d0996ffbfe77 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.591405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105993-d8b4-409e-82b2-d0996ffbfe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.622331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-81105993-d8b4-409e-82b2-d0996ffbfe77 None None] MFA Rules not processed for user `5594b2c1a2fd4bf8ad2f1c5d2c05b777`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:49.623595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-81105993-d8b4-409e-82b2-d0996ffbfe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:49.692684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1192/2376] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:49.703884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:55:49.705806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:49.729679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:55:49.766750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18900-16a9-4c82-9c53-9743c2be7b0a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:55:49.770695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:55:49.771324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:49.771596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:49.771865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:49.780470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:55:49.781896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:55:49.794961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1185/2377] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:55:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:55:51.172989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:55:51.173552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:55:51.173552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:55:51.192916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] MFA Rules not processed for user `5594b2c1a2fd4bf8ad2f1c5d2c05b777`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:55:51.194122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:55:51.201454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1193/2378] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:55:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:55:59.788486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=17 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:01.187959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:06.783736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:06.785510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:06.885489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad8cc2-1123-4141-86c0-6525b5c924fc service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:06.903632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:06.905282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:06.905556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:06.905821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:06.908091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:06.931925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:06.934324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:06.949462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1186/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:10.936315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:10.938010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:10.997165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c0e6c-e568-4d02-8af9-207445a59e75 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:11.000843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:11.001464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:11.001745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:11.002396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:11.006835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:11.084030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:11.085626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:11.097838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1194/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:13.718286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:13.719902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:13.732320 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045cc93-d8ea-476d-a550-c5d389c0bc8d service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:13.736488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:13.737171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:13.737513 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:13.737845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:13.739851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:13.752539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:13.754046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:13.765252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1187/2381] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:13.878571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:13.895282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ac06-6246-4782-9f50-83a624f11f94 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:13.900698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:13.901914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:13.902396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:13.903128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:13.907547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:13.924277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:13.935808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:13.961687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1195/2382] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:14.485101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:14.486922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:14.517716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9eba-db25-4b92-a815-cc69f5d8bf1e service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:14.523435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:14.524048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:14.524861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:14.525188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] PATH_INFO: `/v3/users/31a8c62bd81f4c1f863bd952b6976276` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:14.527968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a8c62bd81f4c1f863bd952b6976276)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:14.531834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:14.589072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a8c62bd81f4c1f863bd952b6976276'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:14.589731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a8c62bd81f4c1f863bd952b6976276'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:14.603209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a8c62bd81f4c1f863bd952b6976276'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:14.611009 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a8c62bd81f4c1f863bd952b6976276'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:14.611753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ac9185d-cf2a-4324-8091-06a3e4c38e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a8c62bd81f4c1f863bd952b6976276'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:14.626584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1188/2383] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:14 2022] DELETE /identity/v3/users/31a8c62bd81f4c1f863bd952b6976276 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:14.893025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:14.908317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48b998-505e-4162-8d9e-c22c5d4fb138 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:14.981865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:14.982747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:14.983068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:14.983404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] PATH_INFO: `/v3/projects/4d80eadc5cc34322b85201c50b3c6552` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:14.984572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d80eadc5cc34322b85201c50b3c6552)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:14.986548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.029613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:15.037920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d80eadc5cc34322b85201c50b3c6552', 'request_id': 'req-b0b06454-2f72-4673-9f09-c82f0a084c36'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:15.046244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d80eadc5cc34322b85201c50b3c6552', 'request_id': 'req-b0b06454-2f72-4673-9f09-c82f0a084c36'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:15.051795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d80eadc5cc34322b85201c50b3c6552', 'request_id': 'req-b0b06454-2f72-4673-9f09-c82f0a084c36'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:15.063988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] The token cache is being invalidate because project 4d80eadc5cc34322b85201c50b3c6552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:15.064274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:15.065056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b0b06454-2f72-4673-9f09-c82f0a084c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:15.066691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1196/2384] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:14 2022] DELETE /identity/v3/projects/4d80eadc5cc34322b85201c50b3c6552 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:15.071739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.072270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.072379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.095590 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:15.096783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:15.139101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:15.161406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1189/2385] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:15.169579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.169867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.170137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.190126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:15.191473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:15.201232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1197/2386] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:15.206237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.218532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147b3b-4ccd-4831-b91d-c6eda94c098a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.224475 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.225099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.225386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.225646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.226313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:15.226688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.228437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.246493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1190/2387] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:15.251130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.261637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e515acd-d758-4085-82cf-5572ea9c3be9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.267084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.267726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.268004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.268277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.268910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.271330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.300918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1198/2388] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:15.305191 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.315969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cac6f-0151-471a-8d69-27d31d787ebd admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.378206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.378832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.379130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.379406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.380039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.381988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.408241 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1191/2389] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:15.409870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.420615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdc832-99bb-417b-b8c8-4540b9870585 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.425621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.426237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.426521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.426796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.427475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.429082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.439637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1199/2390] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:15.443661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.454796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8751cb-d74c-42d2-8bb6-197ffd2b571f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.460154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.460768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.461048 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.461312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.461957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.463618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6185c-0188-4703-a9c5-634a3abb188a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.475374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1192/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:15.480120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.490638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271c1e3-eb72-41fb-8b1b-e8de01f8c215 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.495771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.496430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.496709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.496980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] PATH_INFO: `/v3/projects/30b8bc874a0742b8b48b57f85d12751d/users/d7e94a005a93417294ea17c1a44fc9df/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.512565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b8bc874a0742b8b48b57f85d12751d, user_id=d7e94a005a93417294ea17c1a44fc9df, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.515280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.530124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1200/2392] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:15 2022] PUT /identity/v3/projects/30b8bc874a0742b8b48b57f85d12751d/users/d7e94a005a93417294ea17c1a44fc9df/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:15.540532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.541042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.541898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.571357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] MFA Rules not processed for user `d7e94a005a93417294ea17c1a44fc9df`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:15.572586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:15.639064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1193/2393] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:15.643470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.709078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ecfd1-5764-441f-913c-eb129a27c56a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.713952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.714560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.714834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.715130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.715793 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.717882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.751038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1201/2394] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:15.752046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.766985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc57b4-f30f-4a10-b90b-137ac1663b85 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.819425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:15.839139 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.840118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.840584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.840856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.841526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.843474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.870163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1194/2395] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:15 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:15.871715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.891667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a610e0-6afa-4471-8efc-c68c12ab88cf admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.897034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.898476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.899853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.900331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.901349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.903395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.917988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1202/2396] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:15.922836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.934518 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b4648-bf59-4432-a3ef-6937e974ec8e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:15.939888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:15.940578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:15.940855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:15.941126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] PATH_INFO: `/v3/projects/a5b356433ada453590c2c1ea7c993209/users/b2b689a2905b49d6a202281fb3b05577/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:15.962765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b356433ada453590c2c1ea7c993209, user_id=b2b689a2905b49d6a202281fb3b05577, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:15.967107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:15.983292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1195/2397] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:15 2022] PUT /identity/v3/projects/a5b356433ada453590c2c1ea7c993209/users/b2b689a2905b49d6a202281fb3b05577/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:15.987427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:15.998160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66ba5-3f3a-48fe-abcd-5480a529466f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:16.044951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:16.087585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:16.088365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:16.088777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:16.089137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:16.089989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:16.100625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7a172-1e75-4d86-87c9-1b6b643e8426 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:16.129898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:16.130375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1203/2398] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:16.147350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1e2d-927f-4b1b-8182-92a6fc2a0047 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:16.156351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:16.158215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:16.158761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:16.159268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] PATH_INFO: `/v3/projects/a5b356433ada453590c2c1ea7c993209/users/b2b689a2905b49d6a202281fb3b05577/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:16.163184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b356433ada453590c2c1ea7c993209, user_id=b2b689a2905b49d6a202281fb3b05577, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:16.174411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:16.190187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1196/2399] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:16 2022] PUT /identity/v3/projects/a5b356433ada453590c2c1ea7c993209/users/b2b689a2905b49d6a202281fb3b05577/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:16.203484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:16.203845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:16.204190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:16.258513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] MFA Rules not processed for user `b2b689a2905b49d6a202281fb3b05577`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:16.259862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:16.346570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1204/2400] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:16.364317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:16.366918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:16.435210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196aac4-ba61-43c9-9b6f-c0930df39ef8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:16.439880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:16.440971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:16.441258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:16.441599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:16.451983 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:16.453652 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952a27d-cd7b-4ec6-9b4f-c621ad3aee48 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:16.466004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1197/2401] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:16.569298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:16.584852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac5747-aeed-4fed-803a-26d06a025f4b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:16.590077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:16.590714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:16.591352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:16.591493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:16.653115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:16.654548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:16.665516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1205/2402] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:17.318729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747136-bda9-42dd-867c-234101698a43 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:17.319289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747136-bda9-42dd-867c-234101698a43 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:17.319516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747136-bda9-42dd-867c-234101698a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:17.341784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-bd747136-bda9-42dd-867c-234101698a43 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:17.343307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bd747136-bda9-42dd-867c-234101698a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:17.410322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1198/2403] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:56:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:17.433443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:17.434983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:17.526048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:17.547645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6788e1-e218-4158-9f03-e3e9541e898f service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:17.553214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:17.554124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:17.554536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:17.555569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:17.565480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:17.567429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:17.579784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1206/2404] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:56:18.418119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffe604-7fd1-4d08-bb69-21e1796d3bd3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:18.418698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffe604-7fd1-4d08-bb69-21e1796d3bd3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:18.419133 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffe604-7fd1-4d08-bb69-21e1796d3bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:18.466536 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d7ffe604-7fd1-4d08-bb69-21e1796d3bd3 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:18.468231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d7ffe604-7fd1-4d08-bb69-21e1796d3bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:18.480088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1199/2405] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:56:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:18.499280 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:18.515315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93de850-0134-4213-8682-2d373921c4b5 service swift] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:18.528073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:18.528919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:18.529195 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:18.529460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:18.538493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:18.540435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:18.560287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1207/2406] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:56:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:56:18.864931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094b49-ee19-48d0-bbb7-a86f34e8658f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:18.865756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094b49-ee19-48d0-bbb7-a86f34e8658f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:18.866240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094b49-ee19-48d0-bbb7-a86f34e8658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:18.899682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-83094b49-ee19-48d0-bbb7-a86f34e8658f None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:18.901021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-83094b49-ee19-48d0-bbb7-a86f34e8658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:18.909630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1200/2407] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:56:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:18.921923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:18.935579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33db5-17e8-4fa0-aaf3-9243e85a846b service swift] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:18.943438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:18.944302 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:18.944545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:18.944815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:18.955663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:18.958155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:18.968149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1208/2408] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:56:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:56:19.437985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:19.438578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:19.438578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:19.492851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:19.494079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:19.529861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1201/2409] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:56:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:19.543369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:19.564319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae49155-9a66-4054-8cd6-2d1d4695d410 service swift] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:19.583856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:19.584494 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:19.584773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:19.585041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:19.604206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:19.607304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:19.638308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1209/2410] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:56:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:56:20.014912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:20.019896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.067117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:20.069112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.087949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:20.171731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e0b7c-390d-48a2-a24b-897f8ce9c1c2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:20.181970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:20.182577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.182969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.183385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.189231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.212954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:20.214856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:20.226256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1202/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:20.227270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:20.229542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.239710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1334c8-1f67-4313-99a8-4fbf7ebd05c9 service swift] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:20.246218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b4ee-ae2d-40f1-95f8-0b202b04eed9 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:20.248315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:20.249180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.249937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.250414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] PATH_INFO: `/v3/users/58361634962b4b7884d51be90b683b42` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.251251 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:20.251902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.252177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.252449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] PATH_INFO: `/v3/users/1dde7fad075145c89f726ca85c6c00ba` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.252714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=58361634962b4b7884d51be90b683b42)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:20.253588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dde7fad075145c89f726ca85c6c00ba)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:20.255333 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:20.255543 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:20.319482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58361634962b4b7884d51be90b683b42'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.331982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dde7fad075145c89f726ca85c6c00ba'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.332763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dde7fad075145c89f726ca85c6c00ba'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.333432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58361634962b4b7884d51be90b683b42'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.346403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dde7fad075145c89f726ca85c6c00ba'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.359849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dde7fad075145c89f726ca85c6c00ba'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.360609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dde7fad075145c89f726ca85c6c00ba'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.361612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58361634962b4b7884d51be90b683b42'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.362504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58361634962b4b7884d51be90b683b42'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.363467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58361634962b4b7884d51be90b683b42'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.391750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1210/2412] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:20 2022] DELETE /identity/v3/users/58361634962b4b7884d51be90b683b42 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:20.392056 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1203/2413] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:20 2022] DELETE /identity/v3/users/1dde7fad075145c89f726ca85c6c00ba => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:20.711725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:20.713403 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.737502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:20.739607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.783453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:20.836553 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9eb7-a711-4273-a311-5da965f88e6b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:20.836741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a0bd-45f6-4b72-9ae3-70cdf28cc570 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:20.843034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:20.843499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:20.844121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.844239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.844438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.844790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] PATH_INFO: `/v3/projects/ac9e77355ba040bda4f2186753df7b24` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.845164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.845684 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] PATH_INFO: `/v3/projects/12347de4d2e5465b9a00004084fcb85c` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.846054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9e77355ba040bda4f2186753df7b24)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:20.847842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12347de4d2e5465b9a00004084fcb85c)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:20.848054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:20.849748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:20.877328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:20.905954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12347de4d2e5465b9a00004084fcb85c', 'request_id': 'req-5c22e772-bb1d-4daa-a169-10fc365562b2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.915682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9e77355ba040bda4f2186753df7b24', 'request_id': 'req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.917416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12347de4d2e5465b9a00004084fcb85c', 'request_id': 'req-5c22e772-bb1d-4daa-a169-10fc365562b2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.925144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9e77355ba040bda4f2186753df7b24', 'request_id': 'req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.926262 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12347de4d2e5465b9a00004084fcb85c', 'request_id': 'req-5c22e772-bb1d-4daa-a169-10fc365562b2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.932124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9e77355ba040bda4f2186753df7b24', 'request_id': 'req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.944498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] The token cache is being invalidate because project 12347de4d2e5465b9a00004084fcb85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:20.944979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] The token cache is being invalidate because project ac9e77355ba040bda4f2186753df7b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:20.945275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.945621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.945928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-21f43667-b2a4-4ed0-a69f-fdef1ffbef7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.946640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5c22e772-bb1d-4daa-a169-10fc365562b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:20.947462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1211/2414] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:20 2022] DELETE /identity/v3/projects/ac9e77355ba040bda4f2186753df7b24 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:20.950987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1204/2415] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:20 2022] DELETE /identity/v3/projects/12347de4d2e5465b9a00004084fcb85c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:20.953709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc70a-f266-43d2-9763-a3325d1ea3d1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.954000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc70a-f266-43d2-9763-a3325d1ea3d1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.954274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc70a-f266-43d2-9763-a3325d1ea3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.956343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcfc2-a6e9-4c10-888e-a69cfc46df45 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:20.956630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcfc2-a6e9-4c10-888e-a69cfc46df45 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:20.956968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcfc2-a6e9-4c10-888e-a69cfc46df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:20.977668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-dcabcfc2-a6e9-4c10-888e-a69cfc46df45 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:20.977859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f54bc70a-f266-43d2-9763-a3325d1ea3d1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:20.978931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dcabcfc2-a6e9-4c10-888e-a69cfc46df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:20.979433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f54bc70a-f266-43d2-9763-a3325d1ea3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.046656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1212/2416] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:21.053063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1205/2417] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:21.056224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.056514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.056784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.058899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.059233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.059499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.079887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:21.081227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.083934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:21.085222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.092766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1206/2418] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:21.096042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1213/2419] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:21.098969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.100016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.109327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31cb4f-1316-446d-b86d-18c5292341d9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.110958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558ad21-c834-415f-a789-f7a45835b98c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.113982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.114573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.115046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.115471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.116123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:21.116454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.117772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.118112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.118609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.118962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.119713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.120510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:21.120809 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.122600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.132671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1207/2420] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:56:21.135503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.145560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3007d-c194-44cc-aba7-8a07904f3878 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.147459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1214/2421] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:21.150477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.151209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.151545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.151743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.151945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.152631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.154595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.161902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7851d85-22be-4c29-a940-aa785cb94611 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.166522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.167163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.167435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.167699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.168319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.170212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.183654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1208/2422] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:21.188493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.260989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca15f47-d0d0-4bac-8252-0e16990c3db9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.267434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.268181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.268461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.268757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.269402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.271421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.291680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:21.299101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1209/2423] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:21.302041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.314193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8b120-0757-4066-ba15-a7ea69e575b5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.321284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.321961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.322248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.322523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.323220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.324847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.326425 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1215/2424] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:21.330147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.335676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1210/2425] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:21.339302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.348592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:21.349079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36a39-e235-4bd1-a89b-9ccc76f3692b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.397947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98b87a-21a8-411e-b8c0-cb84d612b5bb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.402826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.403440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.403715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.403981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.404603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.406545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.407033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.407642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.407920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.408186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.408907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.410438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.421395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1211/2426] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:21.425704 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.428621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1216/2427] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:21.432471 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.436007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03986cb1-416f-46b0-9d1b-1df3f7dc9977 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.441237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.441771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df196a8-580f-4c37-bde9-a3d4dd179225 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.441892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.442148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.442436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] PATH_INFO: `/v3/projects/86757f07a99c47d58f35ac91e4609af4/users/d0c66a859ca74390a87d2b2fa9a420e3/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.447272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.447844 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.448116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.448382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.449037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.450753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.457268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86757f07a99c47d58f35ac91e4609af4, user_id=d0c66a859ca74390a87d2b2fa9a420e3, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.460064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff157bf-f6b9-4b14-bf87-f55cf5f4e20d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.460684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1217/2428] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:21.464757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.473803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1212/2429] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:21 2022] PUT /identity/v3/projects/86757f07a99c47d58f35ac91e4609af4/users/d0c66a859ca74390a87d2b2fa9a420e3/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:21.474433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8e79b-3017-4641-94b7-6dac73e261af admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.478568 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.478925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.479607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.499028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:21.500417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.533802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.534414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.534698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.534996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.535631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.537229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.550533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1218/2430] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:21.551568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.558738 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1213/2431] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:21.561862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6ade9-0da0-4bfd-bee2-e61350cdfa40 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.567454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.568102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.568358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.568469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.568669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] PATH_INFO: `/v3/projects/0c500af6dea04db19c4d6e864220c5dd/users/3ab0e446f4a146ceb3b80e8cfa676b41/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.570102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.583906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c500af6dea04db19c4d6e864220c5dd, user_id=3ab0e446f4a146ceb3b80e8cfa676b41, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.586699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a069-0525-43cf-9a38-5dda84323683 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.600421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1219/2432] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:21 2022] PUT /identity/v3/projects/0c500af6dea04db19c4d6e864220c5dd/users/3ab0e446f4a146ceb3b80e8cfa676b41/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:21.610867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.611150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.611422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.628500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566d80e-170e-4f5b-9fc1-9f62d1a7daa2 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.638856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] MFA Rules not processed for user `3ab0e446f4a146ceb3b80e8cfa676b41`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:21.640038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.640264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:21.679204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.679870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.680128 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.680407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.695914 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:21.708181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1220/2433] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:21.719660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:21.722466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:21.753906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.755384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.765916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1214/2434] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:21.808933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b3a7-a2f4-4c2d-bedb-b8078de69856 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:21.815207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:21.815824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:21.816102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:21.816371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:21.824994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:21.826591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7f646-650d-473d-b9e7-e58d39b2c5b7 service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:21.836101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1221/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:22.059170 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.068727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396fe40-9b96-4271-8f33-3b5d567e533d service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.073494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.074127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.074404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.074678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] PATH_INFO: `/v3/users/3ab0e446f4a146ceb3b80e8cfa676b41` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.075840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab0e446f4a146ceb3b80e8cfa676b41)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.077814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.129621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab0e446f4a146ceb3b80e8cfa676b41'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.130359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab0e446f4a146ceb3b80e8cfa676b41'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.167463 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab0e446f4a146ceb3b80e8cfa676b41'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.175767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab0e446f4a146ceb3b80e8cfa676b41'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.177107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab0e446f4a146ceb3b80e8cfa676b41'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.195115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1215/2436] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:22 2022] DELETE /identity/v3/users/3ab0e446f4a146ceb3b80e8cfa676b41 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:22.203227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236513f-54ed-4865-b7c2-1d0a092d1cdf None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.203521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236513f-54ed-4865-b7c2-1d0a092d1cdf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.203801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236513f-54ed-4865-b7c2-1d0a092d1cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.224854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8236513f-54ed-4865-b7c2-1d0a092d1cdf None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:22.226103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8236513f-54ed-4865-b7c2-1d0a092d1cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.293061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1222/2437] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:22.306030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.307770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.356365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:22.371951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60f684-12df-4bca-940a-560d9f72b4bd admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.375566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.376179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.376712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.376712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.386039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.387551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.397505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1216/2438] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:22.486006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.486291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.486599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.518856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] MFA Rules not processed for user `d0c66a859ca74390a87d2b2fa9a420e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:22.520102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.603218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1223/2439] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:22.608071 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.609519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.623300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929878-558c-481d-9655-a6608d312f21 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.628528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.629137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.629416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.629681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.630318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.632345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.662302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1217/2440] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:22.666437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.678123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3dcc-0414-4ab6-8dd7-0e233d2c8d99 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.686530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:22.694378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.696288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.751738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.752355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.752633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.752906 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.753536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.755682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.765461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc2802-4cdb-4944-bdec-301b8004f9d7 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.770888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.771566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.771851 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.772185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] PATH_INFO: `/v3/projects/0c500af6dea04db19c4d6e864220c5dd` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.773434 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c500af6dea04db19c4d6e864220c5dd)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.775377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.779861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1224/2441] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/users => generated 308 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:22.783933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.796571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f1e19-e70c-4de7-9ff3-c8d2ba8ffb3c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.802903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.803411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.803694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.803965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.804704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.806295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.817639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1225/2442] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:22.822513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.833322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34887b-53b7-4b21-833f-71903e0a0491 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:22.837287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c500af6dea04db19c4d6e864220c5dd', 'request_id': 'req-40f31370-b89b-472b-8aa6-ff2e5f0c6690'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.848685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c500af6dea04db19c4d6e864220c5dd', 'request_id': 'req-40f31370-b89b-472b-8aa6-ff2e5f0c6690'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.857266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c500af6dea04db19c4d6e864220c5dd', 'request_id': 'req-40f31370-b89b-472b-8aa6-ff2e5f0c6690'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.869330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] The token cache is being invalidate because project 0c500af6dea04db19c4d6e864220c5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:22.869623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.870285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-40f31370-b89b-472b-8aa6-ff2e5f0c6690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:22.871929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1218/2443] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:22 2022] DELETE /identity/v3/projects/0c500af6dea04db19c4d6e864220c5dd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:22.876686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c3120-c207-49d2-a468-8bf7bf2c5907 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.876974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c3120-c207-49d2-a468-8bf7bf2c5907 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.877255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c3120-c207-49d2-a468-8bf7bf2c5907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.892322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:22.893020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.893301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.893571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] PATH_INFO: `/v3/projects/e9774edf406540268ce71ab9aff51731/users/945b221cf3b048a1897a894b5d00a4fc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.897101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-6f7c3120-c207-49d2-a468-8bf7bf2c5907 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:22.898345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6f7c3120-c207-49d2-a468-8bf7bf2c5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.907149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1219/2444] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:22.912011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9774edf406540268ce71ab9aff51731, user_id=945b221cf3b048a1897a894b5d00a4fc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:22.914863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:22.915014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:22.915275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:22.915569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:22.930838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1226/2445] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:22 2022] PUT /identity/v3/projects/e9774edf406540268ce71ab9aff51731/users/945b221cf3b048a1897a894b5d00a4fc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:22.933550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:22.935116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.935962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:22.937145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:22.978945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:22.998664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77d68-c48a-45f7-b8c3-c1c512809d1b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.001206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1220/2446] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:23.003453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.004190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.004475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.004747 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.005393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.005504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.007091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.014256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb04e5-a6a7-438e-818d-edb2d89e8afc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.018836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1227/2447] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:23.021252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.021833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.022425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.022698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.023027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.023677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:23.023976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.025610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.031192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722123-d461-410e-b3cc-967db4ee7c1b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.036491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.037082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1221/2448] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:23.037207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.037474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.037811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] PATH_INFO: `/v3/projects/e9774edf406540268ce71ab9aff51731/users/945b221cf3b048a1897a894b5d00a4fc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.039915 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9774edf406540268ce71ab9aff51731, user_id=945b221cf3b048a1897a894b5d00a4fc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.041296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.042970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.046501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:23.051213 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d382a0-3b05-4c56-9fbb-aa62f9cc997a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.055868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1228/2449] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:23 2022] PUT /identity/v3/projects/e9774edf406540268ce71ab9aff51731/users/945b221cf3b048a1897a894b5d00a4fc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:23.111835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.112557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.112807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.113074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.113703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.115651 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.148231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1222/2450] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:23.149750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.161960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f5a7-a344-43de-a50c-dc7c63ada25a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.226065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.227220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.227748 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.227971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.228916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.231987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.272870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1229/2451] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:23.277286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.289185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a714-6959-40d9-9bed-0d20bdc75b78 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.294064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.294695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.295039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.295339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.296015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.297629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.310534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1223/2452] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:23.315060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.325876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9ea6-2bb6-4a5f-84ad-f05f070466e5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.330572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.331202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.331480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.331746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.332374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.333965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc1af-2e02-4e6d-a1a0-ec2d71d8c906 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.344521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1230/2453] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:23.349097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.360240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42500-0326-4c1b-b045-2882609fb407 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.365640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.366413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.366672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.367184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] PATH_INFO: `/v3/projects/f512ec6b6c5f4e21816622b4cba0684a/users/b06f4d99b11343efad90848458a7e55f/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.384278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f512ec6b6c5f4e21816622b4cba0684a, user_id=b06f4d99b11343efad90848458a7e55f, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.387922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.405634 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1224/2454] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:23 2022] PUT /identity/v3/projects/f512ec6b6c5f4e21816622b4cba0684a/users/b06f4d99b11343efad90848458a7e55f/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:23.417156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.417431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.417698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.447949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] MFA Rules not processed for user `b06f4d99b11343efad90848458a7e55f`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:23.449375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:23.525230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1231/2455] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:23.529898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.606790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5f61e-446d-4258-9605-dec4bcc9873b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.613874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.614563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.614937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.615316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.616017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.618195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a28be7-e316-409e-992c-fe3a018e4a4f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.647947 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1225/2456] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:23.652041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.663430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e1c23-f3f7-4e2d-beb8-d3224725bc79 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.730083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.730771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.731245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.732798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.734131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.734453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.734788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.736234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.738173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.746533 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:23.753572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:23.768694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] MFA Rules not processed for user `945b221cf3b048a1897a894b5d00a4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:23.770055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:23.770489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1232/2457] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:23.774397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.785459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde55a5-4eee-4ee1-a0be-68ecae7116cb admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.790903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.791611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.791894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.792173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.792903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.795011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.807082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1233/2458] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:23.811274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.822156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213be4a2-aa2d-4d73-93d2-d855ec737b7b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.827088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.827750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.828030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.828299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] PATH_INFO: `/v3/projects/1d22a7a4034e43a0a8769c8562114124/users/48c0bf3449fd430c905ce1e1dbc0f7de/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.844557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d22a7a4034e43a0a8769c8562114124, user_id=48c0bf3449fd430c905ce1e1dbc0f7de, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.847307 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.861602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1226/2459] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:23.865985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1234/2460] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:23 2022] PUT /identity/v3/projects/1d22a7a4034e43a0a8769c8562114124/users/48c0bf3449fd430c905ce1e1dbc0f7de/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:23.870495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.882839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d952f01-5287-4817-96f8-81f9aa3191c3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.953599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.954334 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.954706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.955103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.955850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:23.957606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f91f-a966-4eae-a5eb-ad0c2181481c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:23.970846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1227/2461] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:23.975295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:23.990954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944386-2567-4a0d-af02-4ef50f2e91c8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:23.996214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:23.996911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:23.997179 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:23.997451 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] PATH_INFO: `/v3/projects/1d22a7a4034e43a0a8769c8562114124/users/48c0bf3449fd430c905ce1e1dbc0f7de/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:23.999381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d22a7a4034e43a0a8769c8562114124, user_id=48c0bf3449fd430c905ce1e1dbc0f7de, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:24.002031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:24.024096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1235/2462] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:23 2022] PUT /identity/v3/projects/1d22a7a4034e43a0a8769c8562114124/users/48c0bf3449fd430c905ce1e1dbc0f7de/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:24.026154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.026748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.026898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.058362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] MFA Rules not processed for user `48c0bf3449fd430c905ce1e1dbc0f7de`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:24.059835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.158866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1228/2463] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:24.167268 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:24.169210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.242749 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0b60-74bd-4b40-9080-33e9fa728a18 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:24.246539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:24.247206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.247482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.247752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.262709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:24.264489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3d55-3edb-423f-a73a-362f24bda708 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:24.279149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1236/2464] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:24.312306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:24.313848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.331044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:24.414876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a04d5-3bc8-4460-9e01-74a7364de0f1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:24.419752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:24.420782 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.421077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.421354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] PATH_INFO: `/v3/users/5594b2c1a2fd4bf8ad2f1c5d2c05b777` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.422539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5594b2c1a2fd4bf8ad2f1c5d2c05b777)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:24.425105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:24.525364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5594b2c1a2fd4bf8ad2f1c5d2c05b777'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.526422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5594b2c1a2fd4bf8ad2f1c5d2c05b777'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.549068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5594b2c1a2fd4bf8ad2f1c5d2c05b777'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.557567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5594b2c1a2fd4bf8ad2f1c5d2c05b777'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.558366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5594b2c1a2fd4bf8ad2f1c5d2c05b777'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.577926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1229/2465] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:24 2022] DELETE /identity/v3/users/5594b2c1a2fd4bf8ad2f1c5d2c05b777 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:24.579766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.580074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.580338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.606542 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:24.608010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.684198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1237/2466] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:24.695046 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:24.696563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.786650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970dcaa-b368-4d82-9a52-58d69120a4ea admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:24.791255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:24.792216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.792657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.792963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.804612 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:24.807522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.808500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:24.810338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38f9d9-4959-4e9d-bc0b-5b474082a33b service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:24.820353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e789af4-903b-4a9a-b3d7-0cbf3f16063d None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:24.822733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1230/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:24.825387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:24.826025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.826296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.826569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] PATH_INFO: `/v3/users/b06f4d99b11343efad90848458a7e55f` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.827733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06f4d99b11343efad90848458a7e55f)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:24.829656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:24.878678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:24.881364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06f4d99b11343efad90848458a7e55f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.896180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06f4d99b11343efad90848458a7e55f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.903499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06f4d99b11343efad90848458a7e55f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.907105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06f4d99b11343efad90848458a7e55f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.907785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06f4d99b11343efad90848458a7e55f'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:24.924047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1238/2468] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:24 2022] DELETE /identity/v3/users/b06f4d99b11343efad90848458a7e55f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:24.929176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:24.929562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:24.930594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:24.950157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:24.951428 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:24.987664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:25.033023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1231/2469] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:25.043654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.047664 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.051477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.053356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.075457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aedc2e-0c7e-49b7-b2c3-e6ece5a4407d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.082195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.082867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.083277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.083548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] PATH_INFO: `/v3/projects/78f83c7b81a54c9a936f140ca94e3d34` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.084802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f83c7b81a54c9a936f140ca94e3d34)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.087041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.114144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5ac88-38d0-41b4-8db2-a3b80e23eca1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.119155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.119750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.120049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.120306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.130294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.131905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952501f-dd42-4b3a-be64-2a694492cb75 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.141628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f83c7b81a54c9a936f140ca94e3d34', 'request_id': 'req-54db9e0b-efa1-4d24-8317-9cc52533ae6b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.142398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1239/2470] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:25.151426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f83c7b81a54c9a936f140ca94e3d34', 'request_id': 'req-54db9e0b-efa1-4d24-8317-9cc52533ae6b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.158296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f83c7b81a54c9a936f140ca94e3d34', 'request_id': 'req-54db9e0b-efa1-4d24-8317-9cc52533ae6b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.172861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] The token cache is being invalidate because project 78f83c7b81a54c9a936f140ca94e3d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:25.173550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.174756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-54db9e0b-efa1-4d24-8317-9cc52533ae6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.178907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1232/2471] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/projects/78f83c7b81a54c9a936f140ca94e3d34 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:25.181743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.182034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.182309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.203497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:25.204766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.314317 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1240/2472] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:25.322594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.322982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.323385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.353307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:25.354916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.365723 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1233/2473] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:25.370033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.380571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd068d3-379c-4e69-9981-01f86843e9b7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.385602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.386225 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.386502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.387027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.387682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:25.388541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.390323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.399730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.401924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.408565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1241/2474] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:25.410250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.420311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe24bf-7e2d-4a24-84fd-72c2ad717d2f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.426837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.427491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.427765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.428037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] PATH_INFO: `/v3/users/d0c66a859ca74390a87d2b2fa9a420e3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.429348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715c4c6-bc7a-488c-91d2-873984be8798 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.429488 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c66a859ca74390a87d2b2fa9a420e3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.431468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.435260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.436157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.436443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.436716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.437516 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.441132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.487255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1242/2475] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:25 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:25.497228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c66a859ca74390a87d2b2fa9a420e3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.497893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c66a859ca74390a87d2b2fa9a420e3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.504742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.506757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.514156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c66a859ca74390a87d2b2fa9a420e3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.522255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c66a859ca74390a87d2b2fa9a420e3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.523236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c66a859ca74390a87d2b2fa9a420e3'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.545235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1234/2476] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/users/d0c66a859ca74390a87d2b2fa9a420e3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:25.545235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.546935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.552312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:25.587582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:25.604061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97f0a-0421-4c15-9859-aaa1b703124e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.651246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb8e3a-da56-452c-9ddf-2f18e5e0fb68 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.656760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.657407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.657682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.657959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] PATH_INFO: `/v3/projects/f512ec6b6c5f4e21816622b4cba0684a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.659102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f512ec6b6c5f4e21816622b4cba0684a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.661079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.686130 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.687122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.687427 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.687701 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.688345 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.690690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.719050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1243/2477] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:25 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:25.723039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.727173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f512ec6b6c5f4e21816622b4cba0684a', 'request_id': 'req-b50ed688-f596-48ca-beee-137aefdd7e19'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.738788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982b59-dfe8-4e56-b93e-d6a39de493e6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.739257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f512ec6b6c5f4e21816622b4cba0684a', 'request_id': 'req-b50ed688-f596-48ca-beee-137aefdd7e19'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.748494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f512ec6b6c5f4e21816622b4cba0684a', 'request_id': 'req-b50ed688-f596-48ca-beee-137aefdd7e19'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.761807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] The token cache is being invalidate because project f512ec6b6c5f4e21816622b4cba0684a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:25.762155 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.763658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.768835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1235/2478] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/projects/f512ec6b6c5f4e21816622b4cba0684a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:25.769484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.774015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.819486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.820351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.820599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.820999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.821719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.823586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.834957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1244/2479] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:25.838098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.838098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.849764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed688-f596-48ca-beee-137aefdd7e19 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.850278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092c559-6b8d-47f8-9b67-8d48059b1376 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:25.854895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.855768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:25.856079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.856389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:25.856507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.856665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:25.856936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] PATH_INFO: `/v3/users/48c0bf3449fd430c905ce1e1dbc0f7de` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.857041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] PATH_INFO: `/v3/projects/86757f07a99c47d58f35ac91e4609af4` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:25.858038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c0bf3449fd430c905ce1e1dbc0f7de)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.858200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86757f07a99c47d58f35ac91e4609af4)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:25.860048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.860693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:25.910037 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c0bf3449fd430c905ce1e1dbc0f7de'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.911827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86757f07a99c47d58f35ac91e4609af4', 'request_id': 'req-8b6c2de7-c0ba-4288-aec9-ab788acbc037'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.918488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:25.922529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86757f07a99c47d58f35ac91e4609af4', 'request_id': 'req-8b6c2de7-c0ba-4288-aec9-ab788acbc037'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.925118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c0bf3449fd430c905ce1e1dbc0f7de'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.925959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:25.934646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c0bf3449fd430c905ce1e1dbc0f7de'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.935904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c0bf3449fd430c905ce1e1dbc0f7de'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.936262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c0bf3449fd430c905ce1e1dbc0f7de'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.938762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86757f07a99c47d58f35ac91e4609af4', 'request_id': 'req-8b6c2de7-c0ba-4288-aec9-ab788acbc037'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.951175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1245/2480] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/users/48c0bf3449fd430c905ce1e1dbc0f7de => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:25.952900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] The token cache is being invalidate because project 86757f07a99c47d58f35ac91e4609af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:25.953193 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.953845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:25.955554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.955703 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1236/2481] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/projects/86757f07a99c47d58f35ac91e4609af4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:25.957159 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:25.959207 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:25.960597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.031938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02768fe1-d267-43ed-8cf6-5ac1703bda47 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.032913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c2de7-c0ba-4288-aec9-ab788acbc037 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.037191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.037841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.038138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.038430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.039189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.039353 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.040203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.040574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.040841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.040977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] PATH_INFO: `/v3/users/945b221cf3b048a1897a894b5d00a4fc` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.042378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945b221cf3b048a1897a894b5d00a4fc)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.044849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.058305 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1246/2482] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:26.059518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.061535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.076819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e061a-f0ac-4b5b-9421-e22577d019ad admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.081993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.082743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.083089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.083383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] PATH_INFO: `/v3/projects/b596031cb0e84c3a84c725588e9d3fc8/users/bffcbbc966f94d15a034241239ea1899/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.100443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945b221cf3b048a1897a894b5d00a4fc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.101467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b596031cb0e84c3a84c725588e9d3fc8, user_id=bffcbbc966f94d15a034241239ea1899, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.101617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945b221cf3b048a1897a894b5d00a4fc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.104348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ef988-be71-4cff-bd85-c4623628ce7c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.122681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945b221cf3b048a1897a894b5d00a4fc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.122843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1247/2483] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:26 2022] PUT /identity/v3/projects/b596031cb0e84c3a84c725588e9d3fc8/users/bffcbbc966f94d15a034241239ea1899/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:26.130175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945b221cf3b048a1897a894b5d00a4fc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.130970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945b221cf3b048a1897a894b5d00a4fc'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.133980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.134284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.134551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.151285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1237/2484] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:25 2022] DELETE /identity/v3/users/945b221cf3b048a1897a894b5d00a4fc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:26.163402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] MFA Rules not processed for user `bffcbbc966f94d15a034241239ea1899`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:26.164709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.248817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1248/2485] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:26.256179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.258033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.268049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.273772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.345064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:26.366036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e971a7f-d273-42e8-b79d-adfd18649245 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.369915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275df17-646e-488e-bb89-cd34e4fff8f8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.370535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.371332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.371620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.371893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.375420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.376286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.376788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.377365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] PATH_INFO: `/v3/projects/1d22a7a4034e43a0a8769c8562114124` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.379255 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d22a7a4034e43a0a8769c8562114124)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.381510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.381646 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.383139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.389453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:26.393510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1249/2486] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:26.394269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.396089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.412021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b4865-ecbd-434b-a6bc-edfb1b74adaa service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.418030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.419124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.419575 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.419993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] PATH_INFO: `/v3/projects/e9774edf406540268ce71ab9aff51731` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.421851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9774edf406540268ce71ab9aff51731)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.423976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.438889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d22a7a4034e43a0a8769c8562114124', 'request_id': 'req-95195c18-bf18-479b-958d-d7cb43c3a70e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.447542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d22a7a4034e43a0a8769c8562114124', 'request_id': 'req-95195c18-bf18-479b-958d-d7cb43c3a70e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.453763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d22a7a4034e43a0a8769c8562114124', 'request_id': 'req-95195c18-bf18-479b-958d-d7cb43c3a70e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.465828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] The token cache is being invalidate because project 1d22a7a4034e43a0a8769c8562114124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:26.466135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.466762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-95195c18-bf18-479b-958d-d7cb43c3a70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.468263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1238/2487] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:26 2022] DELETE /identity/v3/projects/1d22a7a4034e43a0a8769c8562114124 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:26.472910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025c46c-644b-4608-bf1c-02bb1d401f59 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.473298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025c46c-644b-4608-bf1c-02bb1d401f59 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.473632 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025c46c-644b-4608-bf1c-02bb1d401f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.483619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9774edf406540268ce71ab9aff51731', 'request_id': 'req-14ccdaa8-83e1-433a-9806-37051a668b98'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.493018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9774edf406540268ce71ab9aff51731', 'request_id': 'req-14ccdaa8-83e1-433a-9806-37051a668b98'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.496554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4025c46c-644b-4608-bf1c-02bb1d401f59 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:26.497769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4025c46c-644b-4608-bf1c-02bb1d401f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.500338 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9774edf406540268ce71ab9aff51731', 'request_id': 'req-14ccdaa8-83e1-433a-9806-37051a668b98'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.512291 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] The token cache is being invalidate because project e9774edf406540268ce71ab9aff51731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:26.512704 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.513416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-14ccdaa8-83e1-433a-9806-37051a668b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:26.514998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1250/2488] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:26 2022] DELETE /identity/v3/projects/e9774edf406540268ce71ab9aff51731 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:26.519911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7020d6-bd82-4fd9-90a4-946b4d34fe1b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.520289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7020d6-bd82-4fd9-90a4-946b4d34fe1b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.520622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7020d6-bd82-4fd9-90a4-946b4d34fe1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.545835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9e7020d6-bd82-4fd9-90a4-946b4d34fe1b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:26.547091 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9e7020d6-bd82-4fd9-90a4-946b4d34fe1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.560529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1239/2489] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:26.568854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.569214 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.569962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.592778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:26.593987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.602053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1240/2490] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:26.606994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.617459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6d16d-da02-4f5a-803e-38d1b4e2c55f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.622046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1251/2491] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:26.630177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.630459 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.630734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.632404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.633135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.633391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.633686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.634384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:26.634691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.636345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.653317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1241/2492] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:56:26.653473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:26.654191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.654514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.663196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1252/2493] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:26.665985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aa89f-3ab9-4b30-b88c-75be5b7fa9f8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.667324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.671752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.672504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.672785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.673052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.673817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.675998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.677251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b30b09-10ca-4026-9be9-ef0a703c14a1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.682386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.683082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.683538 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.683883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.684606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:26.684994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.686690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.709893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1242/2494] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:26.713909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.714474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1253/2495] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:56:26.715376 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.725399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43c5c5-0d9c-4e88-8f46-854fc68de4f6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.787658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:26.796780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:26.802269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41822-7c2e-44e7-813e-7001e5b6e299 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.811848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.812471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.813696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.814278 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.814967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.817411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.827869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.828627 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.828955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.829204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.829919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.832048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.843841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1243/2496] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:26.848622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.858142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ff8-e588-434d-b173-1154e47c70a3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.864683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.865360 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.865701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.866031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.866842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.868570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.871285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1254/2497] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:26.876056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.886247 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1244/2498] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:26.887505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.898256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4da9-21b4-47a8-9a78-56d54465d10e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.957648 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214bb0-d65c-4187-bbdf-94a5851e366f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:26.963904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.964528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.964807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.965078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.965742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.967845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.972073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:26.972742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:26.973080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:26.973413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:26.974118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:26.975814 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:26.987311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1245/2499] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:26.988442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:26.990019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:26.990756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1255/2500] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:26 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:26.994600 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.004978 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fa038-17e5-4274-9dfc-0d4fafd100ce admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.005755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771917-aa92-4686-b55f-623b6166f2f6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.009780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.010419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.010822 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.011086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.011368 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.011792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.012284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.012661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.013040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] PATH_INFO: `/v3/users/d7e94a005a93417294ea17c1a44fc9df` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.013534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.014310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7e94a005a93417294ea17c1a44fc9df)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.016935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.024991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1256/2501] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:27.025848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.035618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70d8e-7608-4441-821e-ff5f1096ca40 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.040206 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.040830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.041103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.041370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] PATH_INFO: `/v3/projects/2955eb93a3b24bdb857d163bffa4d17a/users/6c586f6351fd45c2bb1aa700a602c401/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.056413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2955eb93a3b24bdb857d163bffa4d17a, user_id=6c586f6351fd45c2bb1aa700a602c401, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.059057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.067931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7e94a005a93417294ea17c1a44fc9df'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.068823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7e94a005a93417294ea17c1a44fc9df'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.072596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1257/2502] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:27 2022] PUT /identity/v3/projects/2955eb93a3b24bdb857d163bffa4d17a/users/6c586f6351fd45c2bb1aa700a602c401/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:27.073375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.074904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.087416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7e94a005a93417294ea17c1a44fc9df'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.094593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7e94a005a93417294ea17c1a44fc9df'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.095550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-0f42ee78-a958-4d59-a76f-3342705cac6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7e94a005a93417294ea17c1a44fc9df'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.109540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1246/2503] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:26 2022] DELETE /identity/v3/users/d7e94a005a93417294ea17c1a44fc9df => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:27.110759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc0de-379e-469a-9ff0-aa06a567a8c1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.111075 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc0de-379e-469a-9ff0-aa06a567a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.111354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc0de-379e-469a-9ff0-aa06a567a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.138853 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf7d21-4987-4b56-bd00-27893d0d09cc admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.139414 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-596dc0de-379e-469a-9ff0-aa06a567a8c1 None None] MFA Rules not processed for user `6c586f6351fd45c2bb1aa700a602c401`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:27.140722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-596dc0de-379e-469a-9ff0-aa06a567a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.164382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.169927 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.195366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.195999 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.196273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.196539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.197171 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.199073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.208147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1247/2504] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:27.211871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1258/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:27.212037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.212037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.212037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.212676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.214352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.226859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9ff7-1474-4dbe-9de2-dd1c11ca53e1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.229647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:27.231183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.237103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.237798 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.238146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.238533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.239226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.241092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.243570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1248/2506] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:27.244439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.254522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f406a-d2e8-43fe-aeba-3123c87e0fa1 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.260891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.261560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.261833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.262099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] PATH_INFO: `/v3/projects/154f93861c3546eab982f68ea66f158c/users/e7363c6e316047b787e2befca93cd28e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.274311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1259/2507] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:27.274311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.278736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.278880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] RBAC: Authorizing `identity:create_grant(project_id=154f93861c3546eab982f68ea66f158c, user_id=e7363c6e316047b787e2befca93cd28e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.281498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.295175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1249/2508] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:27 2022] PUT /identity/v3/projects/154f93861c3546eab982f68ea66f158c/users/e7363c6e316047b787e2befca93cd28e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:27.296105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.340560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320c46b-e59d-44ec-a136-d0e5ce1a782e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.364958 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8c7b-b42c-4639-a65b-f8ef6c3eccae admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.370859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.371497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.371775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.372069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.372676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.374617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8a52-f6d0-481a-b5d7-4cb43acf5c94 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.388260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.388922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.389230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.389520 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.396941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1250/2509] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:27.398174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.398486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.398730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.399787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.401382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.412877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1260/2510] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:27.414349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.423913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8649bb-3373-425a-8b29-73c7a349443c service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.428562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] MFA Rules not processed for user `e7363c6e316047b787e2befca93cd28e`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:27.429594 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.429828 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.430201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.430478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.430795 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.431398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.432998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.444122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1261/2511] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:27.448093 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.459259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238afa6-8ce6-4735-923d-7f513fce606b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.465564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.466238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.466561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.466837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] PATH_INFO: `/v3/projects/a6baa394c21b42ad8faf507d74e350f3/users/ef788fa644f54fc289c152d123af8b76/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.483743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6baa394c21b42ad8faf507d74e350f3, user_id=ef788fa644f54fc289c152d123af8b76, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.486700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.502858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1262/2512] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:27 2022] PUT /identity/v3/projects/a6baa394c21b42ad8faf507d74e350f3/users/ef788fa644f54fc289c152d123af8b76/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:27.505905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.515819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1251/2513] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:27.520667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.534148 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e21e1-0731-4e7b-92de-5549a2b930d9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.549961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.550532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.599419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e505a75-2b49-4bf7-ae78-9f57ed7ec100 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.605867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.606534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.606876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.607193 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.607890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.610190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.613185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.613813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.614096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.614377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.615645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.617758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.623898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1263/2514] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:27.628112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.643414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feaf3d-0c33-44a1-80ce-b220a4c80b75 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.648533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.649462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1252/2515] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:27.652650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.652650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.652650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] PATH_INFO: `/v3/projects/a6baa394c21b42ad8faf507d74e350f3/users/ef788fa644f54fc289c152d123af8b76/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.652650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6baa394c21b42ad8faf507d74e350f3, user_id=ef788fa644f54fc289c152d123af8b76, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.653712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.655486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c3d01-d4ed-4bae-8e12-b8aec668e795 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.667473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1264/2516] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:27 2022] PUT /identity/v3/projects/a6baa394c21b42ad8faf507d74e350f3/users/ef788fa644f54fc289c152d123af8b76/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:27.677322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.677547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.677800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.710982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] MFA Rules not processed for user `ef788fa644f54fc289c152d123af8b76`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:27.712314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.727027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289fe8-c7a8-4790-b84f-b4522b6e79b6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.789629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1265/2517] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:27.790483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.791939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.805733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.808220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590bab-509a-4c75-9b18-6ab905372a42 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.809117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.809717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.810550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.811802 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.814408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.815652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.816353 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.816601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.816875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] PATH_INFO: `/v3/projects/30b8bc874a0742b8b48b57f85d12751d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.818248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b8bc874a0742b8b48b57f85d12751d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:27.820217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:27.850995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1253/2518] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:27 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:27.850995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.851535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.877774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b8bc874a0742b8b48b57f85d12751d', 'request_id': 'req-80c48a38-5c46-4869-90c2-32d36b1820fe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.888628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b8bc874a0742b8b48b57f85d12751d', 'request_id': 'req-80c48a38-5c46-4869-90c2-32d36b1820fe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.896123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b8bc874a0742b8b48b57f85d12751d', 'request_id': 'req-80c48a38-5c46-4869-90c2-32d36b1820fe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.904458 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.913189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] The token cache is being invalidate because project 30b8bc874a0742b8b48b57f85d12751d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:27.913609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.914300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:27.918770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1266/2519] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:27 2022] DELETE /identity/v3/projects/30b8bc874a0742b8b48b57f85d12751d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:27.918770 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:27.921505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:27.924739 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e028-40d3-499a-9902-33ff6ec1cf79 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:27.945481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:27.979779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:27.980285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:27.980892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:27.981229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:27.983473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.002609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c48a38-5c46-4869-90c2-32d36b1820fe admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.007843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.008662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.009407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.009715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.010469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.012446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.028732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1267/2520] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:28.028732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.028732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.042153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a31829-55de-4dde-a106-ce9b2998120b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.046907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.047544 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.047823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.048101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] PATH_INFO: `/v3/users/b2b689a2905b49d6a202281fb3b05577` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.049211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b689a2905b49d6a202281fb3b05577)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.051182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.067674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.069140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.079808 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1254/2521] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:28.080599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.090665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871a8af-b1c5-454f-bbec-c4efabbd3ccb service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.096124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.096791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.097064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.097378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] PATH_INFO: `/v3/projects/2542c4fc300b41068daa3e174a16d4eb/users/5b5e777716034362917cedc511391973/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.109511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b689a2905b49d6a202281fb3b05577'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.116248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2542c4fc300b41068daa3e174a16d4eb, user_id=5b5e777716034362917cedc511391973, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.119281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.126644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b689a2905b49d6a202281fb3b05577'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.133859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1255/2522] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:28 2022] PUT /identity/v3/projects/2542c4fc300b41068daa3e174a16d4eb/users/5b5e777716034362917cedc511391973/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:28.135514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b689a2905b49d6a202281fb3b05577'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.136300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b689a2905b49d6a202281fb3b05577'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.136971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b689a2905b49d6a202281fb3b05577'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.138166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.140283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.152925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1268/2523] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:28 2022] DELETE /identity/v3/users/b2b689a2905b49d6a202281fb3b05577 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:28.167963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.169832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.221385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7029767-a72b-49ed-b407-5890d96b5c3a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.246938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90213a-3895-4400-8607-eedb139b3bc3 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.250745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.253464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:28.256435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.256816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.257096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.259141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.291654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.292267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.292540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.292810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.293569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.295225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.307145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:28.310911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1256/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:28.312229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.329114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f062a-2cf4-46bd-a998-90e73aaabaab admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.333971 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.334829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.335507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98cbba-23cc-4dd4-a67d-c9af7c09709a service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.335639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.335823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.336621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] PATH_INFO: `/v3/projects/2542c4fc300b41068daa3e174a16d4eb/users/5b5e777716034362917cedc511391973/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.338646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2542c4fc300b41068daa3e174a16d4eb, user_id=5b5e777716034362917cedc511391973, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.341746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.345789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1269/2525] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:28.361229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1257/2526] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:28 2022] PUT /identity/v3/projects/2542c4fc300b41068daa3e174a16d4eb/users/5b5e777716034362917cedc511391973/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:28.365434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.365985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.366165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.396622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] MFA Rules not processed for user `5b5e777716034362917cedc511391973`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:28.398039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.465256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.467569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.505015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1270/2527] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:28.513517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.563678 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3ed9d-4dfe-419a-b165-3567a944f516 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.568305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.568450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906fd5f-2770-401e-9a9c-09e342d289be None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.568966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.569244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.569517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] PATH_INFO: `/v3/projects/a5b356433ada453590c2c1ea7c993209` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.570711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b356433ada453590c2c1ea7c993209)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.572654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.572791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.573258 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.573531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.573796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.575592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.622225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b356433ada453590c2c1ea7c993209', 'request_id': 'req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.629977 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b356433ada453590c2c1ea7c993209', 'request_id': 'req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.635593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b356433ada453590c2c1ea7c993209', 'request_id': 'req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.640391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.641825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab76cb9-5760-4827-bac7-45d8dabfc07f service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.646607 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] The token cache is being invalidate because project a5b356433ada453590c2c1ea7c993209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:28.646923 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.647586 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-25e15f80-3c88-40eb-80ff-5ca8b79f6b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:28.649357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1258/2528] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:28 2022] DELETE /identity/v3/projects/a5b356433ada453590c2c1ea7c993209 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:28.650751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1271/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:28.654166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.654445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.654716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.674118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:28.675363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.714937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:28.739010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1259/2530] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:28.747432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.747657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.747946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.770169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:28.771561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:28.779995 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1272/2531] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:28.784465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.794901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abace09c-9c20-4950-991b-f704a97fb1be None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.800832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.801577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.801891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.802189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.802872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:28.803177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.804969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.817326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1260/2532] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:56:28.822197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.831968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cdab1-095c-41aa-8f43-f6f5e612b6db None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.836862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.837493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.837751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.838021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.838667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.840638 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.852048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:28.870864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1273/2533] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:28 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:28.873019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:28.899047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4693a-5d4b-474b-a91c-fb715de09dd8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:28.965462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:28.966215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:28.966499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:28.966771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:28.967464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:28.969451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:28.991107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1261/2534] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:28 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:28.995354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.005762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71f32-7590-4184-a243-6187754e7c93 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.010873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.011484 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.013445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.013719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.014371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.016143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.026788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1274/2535] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:29.031132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.042499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78a95-ee70-4623-abd5-e18516cded8c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.047863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.048472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.048751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.049031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.049711 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.051387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7507c52-e87e-4c18-84af-11eda5417e56 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.064699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1262/2536] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:29.069191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.079903 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c77a1-4a10-411a-99dc-9ccc99324818 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.085322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.086010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.086290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.086571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] PATH_INFO: `/v3/projects/218fc979190c4bc0b5a896e0c7f05d21/users/367995cdb8fd4434b771cf6d0d32d11a/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.102328 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218fc979190c4bc0b5a896e0c7f05d21, user_id=367995cdb8fd4434b771cf6d0d32d11a, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.105029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.120693 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1275/2537] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:29 2022] PUT /identity/v3/projects/218fc979190c4bc0b5a896e0c7f05d21/users/367995cdb8fd4434b771cf6d0d32d11a/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:29.132602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.132918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.133181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.163227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] MFA Rules not processed for user `367995cdb8fd4434b771cf6d0d32d11a`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:29.164452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:29.232416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1263/2538] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:29.237691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.310642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4435b89-606f-4894-86d6-88e6f2692c49 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.315547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.316178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.316437 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.316713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.317359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.319327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.352904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1276/2539] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:29 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:29.354861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.368611 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2267b-5c10-4c68-8392-c282d38a58e9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.401312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:29.425649 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.426371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.426650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.426948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.427582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.429581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.453401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1264/2540] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:29 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:29.457562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.468585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8a52-97a4-42c1-8632-c908703178a2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.474348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.474980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.475244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.475512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.476247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.477832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.489277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1277/2541] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:29.493448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.504374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25cf8b-0462-4e23-ad75-fd8d7396b5c4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.510234 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.510825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.511134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.511405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] PATH_INFO: `/v3/projects/21ada22fb4d34a948854107aee99b180/users/c06ba09f44c845fc98b4ad754eeec378/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.526614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ada22fb4d34a948854107aee99b180, user_id=c06ba09f44c845fc98b4ad754eeec378, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.529591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.543863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1265/2542] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:29 2022] PUT /identity/v3/projects/21ada22fb4d34a948854107aee99b180/users/c06ba09f44c845fc98b4ad754eeec378/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:29.547972 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.559082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4170-e1ce-414d-bfad-6796965f48a2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.618256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.618916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.619191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.619455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.620090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.621681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc16b9-8d64-4b07-9052-41a500920fd4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.632107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1278/2543] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:29.636446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.647442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f7b8-0d31-42be-b102-ee3569514752 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.653460 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.654147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.654442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.654698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] PATH_INFO: `/v3/projects/21ada22fb4d34a948854107aee99b180/users/c06ba09f44c845fc98b4ad754eeec378/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.656726 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ada22fb4d34a948854107aee99b180, user_id=c06ba09f44c845fc98b4ad754eeec378, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.659381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.670364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1266/2544] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:29 2022] PUT /identity/v3/projects/21ada22fb4d34a948854107aee99b180/users/c06ba09f44c845fc98b4ad754eeec378/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:29.681718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.682094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.682485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.715877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] MFA Rules not processed for user `c06ba09f44c845fc98b4ad754eeec378`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:29.717088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:29.732870 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:29.792342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1279/2545] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:29.804746 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:29.806836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:29.864889 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0672b9-0b0c-4894-8ef8-f4fde38ac7b8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:29.868971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:29.869580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:29.869856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:29.870125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:29.879904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:29.881351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:29.891187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1267/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:30.013503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:30.024880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532fa7f-36a5-4dac-9a61-249463f73780 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:30.028466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:30.029089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:30.029360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:30.029623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:30.084506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:30.086057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a0d9-a401-4c07-9899-582604d1db2f service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:30.095445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1280/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:31.169589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:31.171159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:31.264980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff72d6-4550-4a91-ad08-3f77530b9118 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:31.272700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:31.273338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:31.273650 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:31.274081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] PATH_INFO: `/v3/users/6c586f6351fd45c2bb1aa700a602c401` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:31.275361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c586f6351fd45c2bb1aa700a602c401)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:31.277404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:31.323967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:31.335485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c586f6351fd45c2bb1aa700a602c401'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.337020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c586f6351fd45c2bb1aa700a602c401'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.358105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c586f6351fd45c2bb1aa700a602c401'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.367236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c586f6351fd45c2bb1aa700a602c401'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.368380 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c586f6351fd45c2bb1aa700a602c401'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.382690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1268/2548] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:31 2022] DELETE /identity/v3/users/6c586f6351fd45c2bb1aa700a602c401 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:31.387456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:31.387993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:31.387993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:31.407397 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:31.408631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:31.471431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1281/2549] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:31.482702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:31.484188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:31.542722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05689d21-86d2-42b2-a23b-9d2d80192c15 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:31.546558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:31.547181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:31.547454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:31.547721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:31.557181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:31.558688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:31.570290 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1269/2550] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:31.859051 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:31.860535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:31.875464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06f178-34ec-440b-8d46-af950b4d1324 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:31.880924 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:31.881560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:31.881836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:31.882105 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] PATH_INFO: `/v3/projects/2955eb93a3b24bdb857d163bffa4d17a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:31.883473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2955eb93a3b24bdb857d163bffa4d17a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:31.885581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:31.933562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:31.936670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2955eb93a3b24bdb857d163bffa4d17a', 'request_id': 'req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.947201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2955eb93a3b24bdb857d163bffa4d17a', 'request_id': 'req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.955752 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2955eb93a3b24bdb857d163bffa4d17a', 'request_id': 'req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.972789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] The token cache is being invalidate because project 2955eb93a3b24bdb857d163bffa4d17a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:31.973106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.973928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:31.976236 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1282/2551] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:31 2022] DELETE /identity/v3/projects/2955eb93a3b24bdb857d163bffa4d17a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:31.981006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:31.991326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ece82-f562-4f06-b09c-4955f679a332 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.051027 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.051844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.052366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.052729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] PATH_INFO: `/v3/users/ef788fa644f54fc289c152d123af8b76` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.054504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef788fa644f54fc289c152d123af8b76)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.057473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.109577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef788fa644f54fc289c152d123af8b76'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.110224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef788fa644f54fc289c152d123af8b76'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.116943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:32.123146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef788fa644f54fc289c152d123af8b76'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.130744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef788fa644f54fc289c152d123af8b76'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.131406 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c7af3c16-bc51-4015-8c10-09f4df84a67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef788fa644f54fc289c152d123af8b76'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.145244 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1270/2552] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:31 2022] DELETE /identity/v3/users/ef788fa644f54fc289c152d123af8b76 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:32.436057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.437543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:32.452487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792942-a2fa-494c-a2f4-14eb7fa0eb03 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.513501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.514219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.514505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.514778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] PATH_INFO: `/v3/projects/a6baa394c21b42ad8faf507d74e350f3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.516000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6baa394c21b42ad8faf507d74e350f3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.517962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.565521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6baa394c21b42ad8faf507d74e350f3', 'request_id': 'req-f723212a-9edd-475d-908b-c5215f128810'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.573441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6baa394c21b42ad8faf507d74e350f3', 'request_id': 'req-f723212a-9edd-475d-908b-c5215f128810'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.579807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6baa394c21b42ad8faf507d74e350f3', 'request_id': 'req-f723212a-9edd-475d-908b-c5215f128810'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.591691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] The token cache is being invalidate because project a6baa394c21b42ad8faf507d74e350f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:32.591974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.592676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f723212a-9edd-475d-908b-c5215f128810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:32.594255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1283/2553] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:32 2022] DELETE /identity/v3/projects/a6baa394c21b42ad8faf507d74e350f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:32.599552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.600029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.600141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.622345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:32.623587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:32.686988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1271/2554] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:32.695981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.696262 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.696533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.716561 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:32.717746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:32.731647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1284/2555] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:32.735991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.746096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4dab5-f500-452a-9e04-6009e6e51efd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.750883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.751505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.751779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.752043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.752702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:32.752995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.754597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.766312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1272/2556] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:32.770583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.780274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5975c-1e43-4040-8d47-76c4f0578d38 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.785094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.785666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.785966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.786235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.786908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.788812 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.820487 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1285/2557] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:32 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:32.824936 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.835033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88efb56-f9cc-405c-ab1e-4d297200657d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.893447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.894044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.894356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.894556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.895252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.897175 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.919199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1273/2558] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:32 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:32.923816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.934461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ffe1-4091-459e-b5ff-14f686c42b06 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.939539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.940147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.940428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.940716 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.941370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.943017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.953319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1286/2559] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:32.957165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:32.967846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63636a3a-bfb8-4196-af87-5d9db7e744d2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:32.972967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:32.973552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:32.973827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:32.974097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:32.974733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:32.976345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d34602-e520-4059-8436-2172f5d6eba1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:32.987609 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1274/2560] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:32.992251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.002312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c47a-ca9b-4f7c-86a4-ced702563944 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.007386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.008016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.008310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.008562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] PATH_INFO: `/v3/projects/20c39dfc7e1349f2a9b1ad7d6a829b45/users/273df3c8bf7a49b7ae3f3801647b8a3e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.022951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c39dfc7e1349f2a9b1ad7d6a829b45, user_id=273df3c8bf7a49b7ae3f3801647b8a3e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.025698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.036486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:33.042128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1287/2561] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:32 2022] PUT /identity/v3/projects/20c39dfc7e1349f2a9b1ad7d6a829b45/users/273df3c8bf7a49b7ae3f3801647b8a3e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:33.052713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.052999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.053267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.080693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] MFA Rules not processed for user `273df3c8bf7a49b7ae3f3801647b8a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:33.081816 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:33.115211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:33.154545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1275/2562] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:33.161079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.225189 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b330f18-8ab9-49cc-9e5e-66e049f707a6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.230190 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.231194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.231497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.231742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.232418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.234360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.263327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1288/2563] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:33 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:33.267578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.278762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2a682-2105-4778-9c67-c525402d0423 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.337413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.338059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.338349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.338617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.339358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.341306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.365261 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1276/2564] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:33 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:33.369276 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.379849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fdf3a-31cb-4816-beb6-1592ecf9ee90 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.385634 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.386077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.386359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.386630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.387300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.389775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.402049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1289/2565] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:33.407053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.418015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4db70-c794-488c-b26c-25e6a1227f89 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.423036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.423719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.423997 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.424269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] PATH_INFO: `/v3/projects/095ff17e124e4c6ba106914075712a54/users/3406077415ef471eb6f66fdd9bd9754e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.439341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095ff17e124e4c6ba106914075712a54, user_id=3406077415ef471eb6f66fdd9bd9754e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.442018 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.459918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1277/2566] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:33 2022] PUT /identity/v3/projects/095ff17e124e4c6ba106914075712a54/users/3406077415ef471eb6f66fdd9bd9754e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:33.464108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.474521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae85f6-fdc0-41f0-8f8e-af619f0952bc admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.535233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.535842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.536149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.536415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.537057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.538709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce68fcf-5959-4f78-b6ae-057b4ffb6de0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.548877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1290/2567] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:33.553321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.564503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288d59d-106b-43cf-9011-8864ffbc7564 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.569597 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.570547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.570714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.571062 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] PATH_INFO: `/v3/projects/095ff17e124e4c6ba106914075712a54/users/3406077415ef471eb6f66fdd9bd9754e/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.573050 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095ff17e124e4c6ba106914075712a54, user_id=3406077415ef471eb6f66fdd9bd9754e, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.575743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.586601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1278/2568] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:33 2022] PUT /identity/v3/projects/095ff17e124e4c6ba106914075712a54/users/3406077415ef471eb6f66fdd9bd9754e/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:33.597310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.597666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.598001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.630157 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] MFA Rules not processed for user `3406077415ef471eb6f66fdd9bd9754e`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:33.631408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:33.699957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:33.714169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1291/2569] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:33.722276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.724354 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:33.784856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa623-7981-4982-a843-de9525c94cb1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.788720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.789340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.789610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.789905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.799669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.801365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae842-991f-470d-8c6a-e516c1d7bdd2 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.811008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1279/2570] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:33.839055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:33.921991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae965355-d993-4f17-8b72-ea89bc58bd0e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:33.930150 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:33.930825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:33.931123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:33.931408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] PATH_INFO: `/v3/users/273df3c8bf7a49b7ae3f3801647b8a3e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:33.932581 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273df3c8bf7a49b7ae3f3801647b8a3e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:33.934577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:33.991001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273df3c8bf7a49b7ae3f3801647b8a3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.004850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273df3c8bf7a49b7ae3f3801647b8a3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.013286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273df3c8bf7a49b7ae3f3801647b8a3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.014039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273df3c8bf7a49b7ae3f3801647b8a3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.014700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273df3c8bf7a49b7ae3f3801647b8a3e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.030156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1292/2571] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:33 2022] DELETE /identity/v3/users/273df3c8bf7a49b7ae3f3801647b8a3e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:34.035057 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:34.035346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:34.035620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:34.054954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:34.056196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:34.075480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:34.125436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1280/2572] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:34.135699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:34.137167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:34.194149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c96e9-e07a-49d0-a8a2-9c266fecd936 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:34.198224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:34.198841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:34.199132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:34.199402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:34.208682 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:34.210126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:34.219712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1293/2573] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:34.377289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:34.378871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:34.442260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e8263-cf18-4252-839e-46f961de2d22 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:34.450235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:34.450858 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:34.451166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:34.451437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:34.453246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:34.524733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:34.526486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:34.536940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1281/2574] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:34.722768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:34.724834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:34.751506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4482fde-7be6-49b0-93f2-dbd041a1fe61 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:34.757020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:34.757754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:34.758143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:34.758473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] PATH_INFO: `/v3/projects/20c39dfc7e1349f2a9b1ad7d6a829b45` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:34.759757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c39dfc7e1349f2a9b1ad7d6a829b45)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:34.761725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:34.774198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:34.823573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c39dfc7e1349f2a9b1ad7d6a829b45', 'request_id': 'req-54dc86f4-23f1-4790-9122-3f9fdea9d986'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.833124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c39dfc7e1349f2a9b1ad7d6a829b45', 'request_id': 'req-54dc86f4-23f1-4790-9122-3f9fdea9d986'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.840791 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c39dfc7e1349f2a9b1ad7d6a829b45', 'request_id': 'req-54dc86f4-23f1-4790-9122-3f9fdea9d986'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.853330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] The token cache is being invalidate because project 20c39dfc7e1349f2a9b1ad7d6a829b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:34.853623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.854269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.856102 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1294/2575] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:34 2022] DELETE /identity/v3/projects/20c39dfc7e1349f2a9b1ad7d6a829b45 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:34.860682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:34.872317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8a17a-d6ce-433e-9b6f-7f4ec7956aa3 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:34.935452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:34.936125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:34.936411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:34.936681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] PATH_INFO: `/v3/users/3406077415ef471eb6f66fdd9bd9754e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:34.937849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3406077415ef471eb6f66fdd9bd9754e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:34.939880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:34.954866 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:34.986020 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3406077415ef471eb6f66fdd9bd9754e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.986663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3406077415ef471eb6f66fdd9bd9754e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:34.998314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3406077415ef471eb6f66fdd9bd9754e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.005681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3406077415ef471eb6f66fdd9bd9754e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.006343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-11ccb404-52a9-4ed7-9f5f-0d8d3c6f10ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3406077415ef471eb6f66fdd9bd9754e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.020789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1282/2576] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:34 2022] DELETE /identity/v3/users/3406077415ef471eb6f66fdd9bd9754e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:35.292566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.294117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:35.308290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc86f4-23f1-4790-9122-3f9fdea9d986 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.371992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.372647 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.372925 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.373177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] PATH_INFO: `/v3/projects/095ff17e124e4c6ba106914075712a54` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.374306 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=095ff17e124e4c6ba106914075712a54)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.376264 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.436464 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095ff17e124e4c6ba106914075712a54', 'request_id': 'req-c75227dd-5087-4425-925b-f65311dae1cc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.444034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095ff17e124e4c6ba106914075712a54', 'request_id': 'req-c75227dd-5087-4425-925b-f65311dae1cc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.449556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095ff17e124e4c6ba106914075712a54', 'request_id': 'req-c75227dd-5087-4425-925b-f65311dae1cc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.461217 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] The token cache is being invalidate because project 095ff17e124e4c6ba106914075712a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:35.461502 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.462160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c75227dd-5087-4425-925b-f65311dae1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:35.463699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1295/2577] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:35 2022] DELETE /identity/v3/projects/095ff17e124e4c6ba106914075712a54 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:35.468791 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.469080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.469362 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.490326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:35.491554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:35.552790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1283/2578] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:35.565942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.566292 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.566642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.588821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:35.590022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:35.599358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1296/2579] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:35.603806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.614381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7c46e-dd42-48a4-bd0b-01a0714ee74f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.620484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.621082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.621359 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.621628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.622285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:35.622588 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.624235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.636680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1284/2580] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:35.641081 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.650721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e95e-8a09-466e-b81c-ee42af36c866 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.656003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.656589 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.656868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.657129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.657765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.659713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.691838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1297/2581] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:35.696970 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.708450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266949b-617b-42f2-8737-d24db5a2b552 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.764859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.765456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.765727 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.765993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.766617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.768618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.792843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1285/2582] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:35.798624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.804608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:35.811027 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c607-e461-4d05-af76-f88df1cce336 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.816847 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.817442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.817721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.817987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.818631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.820358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.830314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1298/2583] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:35.834527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.844920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f4edf-3a22-4402-bb67-34fc963eae69 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.849689 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.850295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.850558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.850859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.851496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.853085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d83451-97a3-4b85-9167-52bd5e7e22fb admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.860343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:35.867250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1286/2584] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:35.871428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:35.882086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bbf4c-6937-4836-a58c-ff4658589f04 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:35.887566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:35.888275 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.888796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.889073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] PATH_INFO: `/v3/projects/f7b373816c6540d392f885f90a8e8efb/users/e0150370e0ef448280dfa2ec72717914/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.904413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b373816c6540d392f885f90a8e8efb, user_id=e0150370e0ef448280dfa2ec72717914, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:35.907211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:35.921371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1299/2585] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:35 2022] PUT /identity/v3/projects/f7b373816c6540d392f885f90a8e8efb/users/e0150370e0ef448280dfa2ec72717914/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:35.931871 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:35.932157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:35.932429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:35.960235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] MFA Rules not processed for user `e0150370e0ef448280dfa2ec72717914`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:35.961502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:36.026581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1287/2586] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:36.032868 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.097491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c52c-474c-4286-9112-71641a868bf6 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.103237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.103846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.104127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.104380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.105011 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.107301 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.136695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1300/2587] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:36.141095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.152151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf92c8c-9dc8-4e05-af77-072af65a49bc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.211990 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.212550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.212832 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.213106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.213759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.215750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.238447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1288/2588] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:36 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:36.242668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.254015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910f054-938c-4991-b20b-0d25aea9dac0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.259645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.260257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.260529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.260797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.261468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.263100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.274016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1301/2589] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:36.278466 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.288707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba0a6-9519-483f-9641-31ace5f336b7 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.294464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.295151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.295462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.295751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] PATH_INFO: `/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3/users/65276a770ac94e9d864b0c6fff9f0487/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.311499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa18fd971ff4f0094815e3a3df1a5c3, user_id=65276a770ac94e9d864b0c6fff9f0487, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.314152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.328716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1289/2590] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:36 2022] PUT /identity/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3/users/65276a770ac94e9d864b0c6fff9f0487/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:36.332713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.342671 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae00c8-fba8-441d-996e-3fe960523fc1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.406777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.407414 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.407690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.407955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.408567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.410172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952e660-d3a2-4f4d-a3b6-5373027cc60c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.419780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1302/2591] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:36.424530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.436022 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291d35-8dbe-40d0-af4a-fdd385720ad6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.441698 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.442357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.442637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.442961 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] PATH_INFO: `/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3/users/65276a770ac94e9d864b0c6fff9f0487/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.445055 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa18fd971ff4f0094815e3a3df1a5c3, user_id=65276a770ac94e9d864b0c6fff9f0487, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.447861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.458239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1290/2592] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:36 2022] PUT /identity/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3/users/65276a770ac94e9d864b0c6fff9f0487/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:36.468525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6582-3768-425e-87b6-6827346eb966 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.468779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6582-3768-425e-87b6-6827346eb966 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.469080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6582-3768-425e-87b6-6827346eb966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.497778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-66db6582-3768-425e-87b6-6827346eb966 None None] MFA Rules not processed for user `65276a770ac94e9d864b0c6fff9f0487`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:36.499042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-66db6582-3768-425e-87b6-6827346eb966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:36.525040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:36.567476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1303/2593] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:36.578785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:36.580895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:36.630691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:36.643577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a203ad-8386-4d13-8f7a-975b2dd4e152 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:36.647540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:36.648135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:36.648441 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:36.648691 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:36.658084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:36.659600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a45079-676c-407f-b31d-f8269e0ca7fe service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:36.670087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1291/2594] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:37.244163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db6582-3768-425e-87b6-6827346eb966 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:37.307579 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db6582-3768-425e-87b6-6827346eb966 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:37.312629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:37.313259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:37.313531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:37.313797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] PATH_INFO: `/v3/users/e0150370e0ef448280dfa2ec72717914` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:37.315040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0150370e0ef448280dfa2ec72717914)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:37.316934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:37.372609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0150370e0ef448280dfa2ec72717914'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.388330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0150370e0ef448280dfa2ec72717914'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.395907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0150370e0ef448280dfa2ec72717914'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.396676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0150370e0ef448280dfa2ec72717914'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.397294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0150370e0ef448280dfa2ec72717914'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.413441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1304/2595] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:37 2022] DELETE /identity/v3/users/e0150370e0ef448280dfa2ec72717914 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:37.418336 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:37.418869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:37.418973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:37.443394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:37.444604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:37.511499 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1292/2596] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:37.521540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:37.522992 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:37.571039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:37.586012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1affc-dd97-4c1a-9b8e-3d36cb2ac382 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:37.589725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:37.590476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:37.590762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:37.591073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:37.600334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:37.601773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:37.611912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1305/2597] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:37.903409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:37.904959 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:37.920129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbb784-ce12-479b-8c15-3e99d98a99f7 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:37.925884 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:37.926558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:37.926844 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:37.927166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] PATH_INFO: `/v3/projects/f7b373816c6540d392f885f90a8e8efb` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:37.928436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b373816c6540d392f885f90a8e8efb)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:37.930349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:37.979250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b373816c6540d392f885f90a8e8efb', 'request_id': 'req-083a7b69-f698-4ca7-9680-c79e0f7a8e36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:37.991519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b373816c6540d392f885f90a8e8efb', 'request_id': 'req-083a7b69-f698-4ca7-9680-c79e0f7a8e36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.001694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b373816c6540d392f885f90a8e8efb', 'request_id': 'req-083a7b69-f698-4ca7-9680-c79e0f7a8e36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.019060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] The token cache is being invalidate because project f7b373816c6540d392f885f90a8e8efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:38.019552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.020219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.021916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1293/2598] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:37 2022] DELETE /identity/v3/projects/f7b373816c6540d392f885f90a8e8efb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:38.025912 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.036168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dceb3d-5de5-4e33-92d2-7b25b3c0690c service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:38.092883 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:38.093432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.093707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.093977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] PATH_INFO: `/v3/users/65276a770ac94e9d864b0c6fff9f0487` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.095194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65276a770ac94e9d864b0c6fff9f0487)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:38.097110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:38.147735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65276a770ac94e9d864b0c6fff9f0487'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.162837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65276a770ac94e9d864b0c6fff9f0487'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.174269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65276a770ac94e9d864b0c6fff9f0487'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.175460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65276a770ac94e9d864b0c6fff9f0487'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.176274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2be92f6c-cca7-4633-a4f1-457ae33ac8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65276a770ac94e9d864b0c6fff9f0487'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.193022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1306/2599] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:38 2022] DELETE /identity/v3/users/65276a770ac94e9d864b0c6fff9f0487 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:38.481455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.483095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:38.496665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7b69-f698-4ca7-9680-c79e0f7a8e36 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:38.554758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:38.555447 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.555728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.556005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] PATH_INFO: `/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.557153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa18fd971ff4f0094815e3a3df1a5c3)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:38.559183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:38.570054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:38.613036 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa18fd971ff4f0094815e3a3df1a5c3', 'request_id': 'req-e89825c5-6d03-41e8-ade9-7dd03d806f9a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.621264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa18fd971ff4f0094815e3a3df1a5c3', 'request_id': 'req-e89825c5-6d03-41e8-ade9-7dd03d806f9a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.627066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa18fd971ff4f0094815e3a3df1a5c3', 'request_id': 'req-e89825c5-6d03-41e8-ade9-7dd03d806f9a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.638292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] The token cache is being invalidate because project 8fa18fd971ff4f0094815e3a3df1a5c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:38.638579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.639314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e89825c5-6d03-41e8-ade9-7dd03d806f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:38.640982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1294/2600] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:38 2022] DELETE /identity/v3/projects/8fa18fd971ff4f0094815e3a3df1a5c3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:38.646098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.646593 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.646687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.666625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:38.667877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:38.726928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1307/2601] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:38.735242 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.735531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.735810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.756422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:38.757630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:38.767954 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1295/2602] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:38.772498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.779252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:38.783389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be742-8d73-4de3-94b2-e9a086a1fc55 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:38.790792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:38.791453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.791927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.792243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.793036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:38.793382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:38.795382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:38.811019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1308/2603] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:38.815721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.825473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf777-784d-44bb-aed2-268f5925fb98 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:38.831549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:38.832153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.832424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.832696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.833332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:38.835296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:38.867971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1296/2604] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:38 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:38.872501 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.882563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93f6d-c7d8-4c21-b407-0983d54a6593 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:38.954562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:38.955229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:38.955509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:38.955772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:38.956428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:38.958344 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:38.981568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1309/2605] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:38 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:38.985666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:38.996338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1e079-8e06-4488-a518-d2ffafa42404 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.001305 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.001907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.002187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.002462 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.003141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.004797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.015256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1297/2606] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:39.019234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.029955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd089f-383c-4639-9a2b-88f0c558ba5e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.034921 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.035525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.035804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.036069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.036710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.038315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d84882-c654-4b64-9d2d-7da4242728fe admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.056476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1310/2607] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:39.060688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.071285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae80bea-89a8-46cc-a393-7d2e160060d1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.076880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.077539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.077821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.078091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] PATH_INFO: `/v3/projects/252dc8d2d20c488ebcb9dffa8d59e070/users/2a5b1b14955e40e2a05417e340e6d07e/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.093842 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252dc8d2d20c488ebcb9dffa8d59e070, user_id=2a5b1b14955e40e2a05417e340e6d07e, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.096559 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.111404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1298/2608] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:39 2022] PUT /identity/v3/projects/252dc8d2d20c488ebcb9dffa8d59e070/users/2a5b1b14955e40e2a05417e340e6d07e/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:39.122135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.122327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.122603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.150772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] MFA Rules not processed for user `2a5b1b14955e40e2a05417e340e6d07e`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:39.152076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:39.218450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1311/2609] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:39.223100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.287079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088710-6a4e-4140-9ce9-4928ce843993 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.293292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.293896 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.294168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.294435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.295119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.297112 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.327098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1299/2610] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:39 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:39.332993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.345842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c066242-e698-4d37-882d-2cd8e148536a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.406216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.406751 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.407170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.407421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.408066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.410012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.418460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:39.433973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1312/2611] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:39 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:39.438196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.448585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4671a1-4cdb-447c-96b7-7270a0787bea admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.455550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.456160 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.456444 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.456745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.457397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.459061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.469887 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1300/2612] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:39.474198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.486862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96abf-1dcf-416a-9ba2-1a94311ff50f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.495622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.496609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.497004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.497348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] PATH_INFO: `/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012/users/fef4c5f30d944cf89f8102ddc18da3dc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.517938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c5187e0aa8444d95b9df4b1f5ff012, user_id=fef4c5f30d944cf89f8102ddc18da3dc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.520670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.536835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1313/2613] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:39 2022] PUT /identity/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012/users/fef4c5f30d944cf89f8102ddc18da3dc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:39.541073 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.551732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf23c5a-4a1b-42e2-b366-0e4afb0736ea admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.557082 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:39.615125 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.615744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.616049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.616322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.616967 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.618610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560750-4ac2-4e61-b789-493b2b487b68 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.629440 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1301/2614] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:39.634015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.644750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafef096-1c8f-4ae7-9252-1117c5d279e4 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.649882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.650539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.650957 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.651231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] PATH_INFO: `/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012/users/fef4c5f30d944cf89f8102ddc18da3dc/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.653207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c5187e0aa8444d95b9df4b1f5ff012, user_id=fef4c5f30d944cf89f8102ddc18da3dc, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.655901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.666354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1314/2615] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:39 2022] PUT /identity/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012/users/fef4c5f30d944cf89f8102ddc18da3dc/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:39.735642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.735935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.736196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.767485 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] MFA Rules not processed for user `fef4c5f30d944cf89f8102ddc18da3dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:39.768712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:39.842303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1302/2616] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:39.854622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:39.858058 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:39.922407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b36f7-3110-485a-8790-986ec1d080b2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:39.926711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:39.927374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:39.927656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:39.927959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:39.937819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:39.939299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:39.949135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1315/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:40.051194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:40.064257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495592f9-f4cf-442c-be57-70deaf612bfe None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:40.068553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:40.069178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:40.069459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:40.069761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:40.138052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:40.140077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40bd23-29dd-4a44-ad04-f08491224906 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:40.152137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1303/2618] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:40.446175 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:40.511325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4744b-a854-48ad-9224-163f2b9b1381 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:40.516934 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:40.517566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:40.517845 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:40.518122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] PATH_INFO: `/v3/users/2a5b1b14955e40e2a05417e340e6d07e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:40.519365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5b1b14955e40e2a05417e340e6d07e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:40.521351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:40.577219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5b1b14955e40e2a05417e340e6d07e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:40.587405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:40.594285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5b1b14955e40e2a05417e340e6d07e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:40.602678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5b1b14955e40e2a05417e340e6d07e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:40.603511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5b1b14955e40e2a05417e340e6d07e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:40.604181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5b1b14955e40e2a05417e340e6d07e'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:40.618533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1316/2619] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:40 2022] DELETE /identity/v3/users/2a5b1b14955e40e2a05417e340e6d07e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:40.623469 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:40.624010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:40.624010 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:40.644052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:40.645295 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:40.669952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:40.705628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1304/2620] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:40.715994 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:40.717442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:40.776229 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4aaf-5d5d-4a3d-84e9-2c2d66716d56 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:40.780209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:40.780829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:40.781103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:40.781365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:40.790585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:40.792219 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:40.802541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1317/2621] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:41.076378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:41.077908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:41.090369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf585d-d3d4-4215-98e6-c2e9738b39fd None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:41.095600 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:41.096256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.096535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.096792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] PATH_INFO: `/v3/projects/252dc8d2d20c488ebcb9dffa8d59e070` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.098598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252dc8d2d20c488ebcb9dffa8d59e070)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:41.100656 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:41.149402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252dc8d2d20c488ebcb9dffa8d59e070', 'request_id': 'req-e3e02100-9670-4964-b8b8-7bd7bad01cf5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.157479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252dc8d2d20c488ebcb9dffa8d59e070', 'request_id': 'req-e3e02100-9670-4964-b8b8-7bd7bad01cf5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.163287 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252dc8d2d20c488ebcb9dffa8d59e070', 'request_id': 'req-e3e02100-9670-4964-b8b8-7bd7bad01cf5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.174752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] The token cache is being invalidate because project 252dc8d2d20c488ebcb9dffa8d59e070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:41.175232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.175984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.177633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1305/2622] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:41 2022] DELETE /identity/v3/projects/252dc8d2d20c488ebcb9dffa8d59e070 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:41.182126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:41.192186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d70e1-7ece-49b1-9404-f99cd5de062f service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:41.261178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:41.261835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.262116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.262412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] PATH_INFO: `/v3/users/fef4c5f30d944cf89f8102ddc18da3dc` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.264508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef4c5f30d944cf89f8102ddc18da3dc)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:41.266774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:41.318601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef4c5f30d944cf89f8102ddc18da3dc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.331111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef4c5f30d944cf89f8102ddc18da3dc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.337784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef4c5f30d944cf89f8102ddc18da3dc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.338588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef4c5f30d944cf89f8102ddc18da3dc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.339233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-c4405e75-0a7f-4c58-9c65-758d1c71b230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef4c5f30d944cf89f8102ddc18da3dc'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.342835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:41.355846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1318/2623] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:41 2022] DELETE /identity/v3/users/fef4c5f30d944cf89f8102ddc18da3dc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:41.642223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:41.643715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:41.656841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e02100-9670-4964-b8b8-7bd7bad01cf5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:41.716116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:41.716785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.717042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.717313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] PATH_INFO: `/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.718446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c5187e0aa8444d95b9df4b1f5ff012)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:41.720405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:41.777040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c5187e0aa8444d95b9df4b1f5ff012', 'request_id': 'req-c9d6d177-51a5-44d3-a42d-3075e31981ee'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.780385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:41.786415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c5187e0aa8444d95b9df4b1f5ff012', 'request_id': 'req-c9d6d177-51a5-44d3-a42d-3075e31981ee'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.792550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c5187e0aa8444d95b9df4b1f5ff012', 'request_id': 'req-c9d6d177-51a5-44d3-a42d-3075e31981ee'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.804534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] The token cache is being invalidate because project d6c5187e0aa8444d95b9df4b1f5ff012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:41.804821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.805527 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-c9d6d177-51a5-44d3-a42d-3075e31981ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:41.807190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1306/2624] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:41 2022] DELETE /identity/v3/projects/d6c5187e0aa8444d95b9df4b1f5ff012 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:41.812300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.812846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.812846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.833904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:41.835141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:41.898152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1319/2625] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:41.906338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.906618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.906924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.933096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:41.934350 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:41.943907 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1307/2626] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:41.948698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:41.959413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9ec11-94cb-4c93-894a-b8247180c7cb None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:41.965327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:41.965931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:41.966212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:41.966490 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:41.967198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:41.967506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:41.969147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:41.982554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1320/2627] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:41.987431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:41.999662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201ed2-5ac0-41d3-881a-261ce1b679aa None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.005529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.006130 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.006438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.006712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.007383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.009308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.044847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1308/2628] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:42.051541 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.064630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c24f-fb3d-4fd2-8de9-c8bb1157a131 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.127350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.128107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.128386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.128655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.129295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.131251 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.154657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1321/2629] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:42 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:42.159198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.170115 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d2be6-f5e6-48ad-abfb-0ed40cde0878 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.175902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.176491 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.176767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.177035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.177671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.179325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.189525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1309/2630] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:42.193595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.204092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556df52-c06b-4506-900c-28046e21221a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.209519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.210127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.210405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.210675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.211406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.213075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42e619-12ee-489b-8a81-d4b470eae895 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.224151 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1322/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:42.228589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.239963 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d4dc-1e8f-4408-8a72-b04865b2d1cc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.245452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.246127 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.246404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.246697 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] PATH_INFO: `/v3/projects/ada8cc87921245a5957e8ac1f6f5f084/users/56d24ea01ce64f7ab0bde83cf136866a/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.262226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada8cc87921245a5957e8ac1f6f5f084, user_id=56d24ea01ce64f7ab0bde83cf136866a, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.264957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.280209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1310/2632] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:42 2022] PUT /identity/v3/projects/ada8cc87921245a5957e8ac1f6f5f084/users/56d24ea01ce64f7ab0bde83cf136866a/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:42.290567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.291064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.291412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.320618 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] MFA Rules not processed for user `56d24ea01ce64f7ab0bde83cf136866a`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:42.321958 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:42.374196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:42.391997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1323/2633] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:42.403274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.404969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:42.473069 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4dd1-f7b7-425b-878c-11c6a141a553 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.479965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.480943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.481211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.481452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.490968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.492402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3729ce-f390-4d11-ad89-42a9fd9d8918 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.501432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1311/2634] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:42.523660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.591765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada612-0f0e-4424-9413-4a4d5314c79c None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.596753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.597389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.597673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.597946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] PATH_INFO: `/v3/users/56d24ea01ce64f7ab0bde83cf136866a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.599146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d24ea01ce64f7ab0bde83cf136866a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.601090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.656885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d24ea01ce64f7ab0bde83cf136866a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:42.669608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d24ea01ce64f7ab0bde83cf136866a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:42.676539 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d24ea01ce64f7ab0bde83cf136866a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:42.677272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d24ea01ce64f7ab0bde83cf136866a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:42.677898 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d24ea01ce64f7ab0bde83cf136866a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:42.692683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1324/2635] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:42 2022] DELETE /identity/v3/users/56d24ea01ce64f7ab0bde83cf136866a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:42.697531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.698028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.698028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.719391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:42.720641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:42.786225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1312/2636] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:42.798405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:42.800116 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:42.858399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e62aa-0880-4864-a44d-4124dc2500d8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:42.862135 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:42.862754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:42.863067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:42.863340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:42.873789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:42.875263 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98f1b-25b3-4ead-b0b7-5e09a7e8ef58 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:42.884584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1325/2637] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:43.169077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.170519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.184477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e957d4-5db2-463a-9749-0458375cbe0d None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.190754 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.191729 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.192029 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.192346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] PATH_INFO: `/v3/projects/ada8cc87921245a5957e8ac1f6f5f084` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.193495 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada8cc87921245a5957e8ac1f6f5f084)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.195633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.213202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:43.245591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada8cc87921245a5957e8ac1f6f5f084', 'request_id': 'req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.259674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada8cc87921245a5957e8ac1f6f5f084', 'request_id': 'req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.265766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada8cc87921245a5957e8ac1f6f5f084', 'request_id': 'req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.277297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] The token cache is being invalidate because project ada8cc87921245a5957e8ac1f6f5f084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:43.277585 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.278347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.279895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1313/2638] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:43 2022] DELETE /identity/v3/projects/ada8cc87921245a5957e8ac1f6f5f084 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:43.284918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb072-fa68-4c07-89c6-f6e9661a1a1d None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.285198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb072-fa68-4c07-89c6-f6e9661a1a1d None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.285468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb072-fa68-4c07-89c6-f6e9661a1a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.305560 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2b1eb072-fa68-4c07-89c6-f6e9661a1a1d None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:43.306772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2b1eb072-fa68-4c07-89c6-f6e9661a1a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.316055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:43.347413 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.350328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.376718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1326/2639] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:43.385886 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.386187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.386458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.408090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:43.409568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.419426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1327/2640] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:43.424044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.427078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2b87b-c4da-4c70-ad9e-430caebe8dc9 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.433330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.433760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b0ff-ccbd-470b-9ad3-443f1681eef9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.434042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.434289 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.434683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] PATH_INFO: `/v3/users/367995cdb8fd4434b771cf6d0d32d11a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.435827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367995cdb8fd4434b771cf6d0d32d11a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.437835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.439007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.439613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.439889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.440154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.440848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:43.441211 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.442799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.455537 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1328/2641] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:43.463909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.474703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac04b-ed6b-4299-99c5-a922334113f1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.479938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.480996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.481365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.481852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.482534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.485022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.502393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367995cdb8fd4434b771cf6d0d32d11a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.504001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367995cdb8fd4434b771cf6d0d32d11a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.518010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1329/2642] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:43.521257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367995cdb8fd4434b771cf6d0d32d11a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.522456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.524786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.529426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367995cdb8fd4434b771cf6d0d32d11a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.530394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-fcd93d15-6085-4780-af46-c1474d188c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367995cdb8fd4434b771cf6d0d32d11a'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:43.548879 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1314/2643] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:43 2022] DELETE /identity/v3/users/367995cdb8fd4434b771cf6d0d32d11a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:43.553687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.554106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.555152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.575850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:43.577194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.596063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:43.601529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99cc26-02e5-430c-a159-283cf004baff admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.662211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1315/2644] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:43.672072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.673561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.675007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.675637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.675910 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.676186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.676978 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.678941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.700993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1330/2645] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:43.705690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.716480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b87866-bfd7-4b8c-a6a9-6dc5fcaf7db7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.721137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.721738 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.722018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.722296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.723418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.725060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.727335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:43.734736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca603-58ad-44cf-9f45-25ca32b526d7 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.738529 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.739031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.739386 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.739682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.741696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1331/2646] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:43.742571 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.749562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.751042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.752372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a6f38-9e3c-4a65-b3b3-507f45bdb5d9 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.757233 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.757846 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.758147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.758450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.759161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.760563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1316/2647] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:43.760824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cca223-b746-4c17-a446-752cf4c442ef admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.772044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1332/2648] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:43.775841 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:43.785497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61556672-ec5a-4d34-a61a-6502b232be21 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:43.790326 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:43.791017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.791300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.791537 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] PATH_INFO: `/v3/projects/2b3535b850f14e30b5cd40e1ba75f8e3/users/c9d344ee1daf4c9c8e9abeffd4b3ce45/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.809063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3535b850f14e30b5cd40e1ba75f8e3, user_id=c9d344ee1daf4c9c8e9abeffd4b3ce45, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:43.811822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:43.826525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1317/2649] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:43 2022] PUT /identity/v3/projects/2b3535b850f14e30b5cd40e1ba75f8e3/users/c9d344ee1daf4c9c8e9abeffd4b3ce45/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:43.837167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:43.837455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:43.837723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:43.869098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] MFA Rules not processed for user `c9d344ee1daf4c9c8e9abeffd4b3ce45`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:43.870369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:43.963032 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1333/2650] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:43.967706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.052174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3545bb-a8ed-4412-9721-acbcf0020fcc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.054621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.056355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.058109 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.058893 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.059627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.059905 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.060556 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.062837 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.071913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e5f8-71f9-4bba-89f6-7208606d7777 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.076816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.077470 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.077781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.078056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] PATH_INFO: `/v3/projects/218fc979190c4bc0b5a896e0c7f05d21` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.079446 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=218fc979190c4bc0b5a896e0c7f05d21)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.081744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.098606 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1318/2651] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:43 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:44.103567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.141256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:44.155295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218fc979190c4bc0b5a896e0c7f05d21', 'request_id': 'req-b271091a-a4a3-4fed-8115-06ad43626953'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.163817 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218fc979190c4bc0b5a896e0c7f05d21', 'request_id': 'req-b271091a-a4a3-4fed-8115-06ad43626953'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.169532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218fc979190c4bc0b5a896e0c7f05d21', 'request_id': 'req-b271091a-a4a3-4fed-8115-06ad43626953'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.184110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] The token cache is being invalidate because project 218fc979190c4bc0b5a896e0c7f05d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:44.184675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.185522 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.190296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1334/2652] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:44 2022] DELETE /identity/v3/projects/218fc979190c4bc0b5a896e0c7f05d21 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:44.190453 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a1c5d-66b3-4478-80ae-5e26f7a92dc4 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.191343 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.192777 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.267030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.267443 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.267719 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.267988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.268626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.270714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.285368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271091a-a4a3-4fed-8115-06ad43626953 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.291740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.292400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.292932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.293227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] PATH_INFO: `/v3/users/c06ba09f44c845fc98b4ad754eeec378` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.294394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06ba09f44c845fc98b4ad754eeec378)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.296397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1319/2653] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:44 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:44.296518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.298069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:44.300833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.310113 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.315492 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c988b-f558-4631-9c7d-0fe319ca9543 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.321836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.322541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.322945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.323330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.324084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.326226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.341376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1320/2654] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:44.342820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.356014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06ba09f44c845fc98b4ad754eeec378'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.363984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637e3a-5d00-4c4b-9b8d-b4c047189a2c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.368854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.369845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.369845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.370301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] PATH_INFO: `/v3/projects/cc38c9208bfe4cda947eb8968e5bf152/users/71cc64589e43465c880ece8afcc03a17/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.371952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06ba09f44c845fc98b4ad754eeec378'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.381294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06ba09f44c845fc98b4ad754eeec378'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.382234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06ba09f44c845fc98b4ad754eeec378'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.382907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06ba09f44c845fc98b4ad754eeec378'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.388891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc38c9208bfe4cda947eb8968e5bf152, user_id=71cc64589e43465c880ece8afcc03a17, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.391660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.405823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1321/2655] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:44 2022] PUT /identity/v3/projects/cc38c9208bfe4cda947eb8968e5bf152/users/71cc64589e43465c880ece8afcc03a17/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:44.408837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1335/2656] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:44 2022] DELETE /identity/v3/users/c06ba09f44c845fc98b4ad754eeec378 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:44.409788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.411505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.490614 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe793c-4401-4f70-9756-a899cb036747 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.537894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:44.571179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.571944 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.582101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.583203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.584618 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.587164 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b67c6-6def-468c-83e6-17d1c78bde94 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.602352 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1322/2657] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:56:44.603545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.614880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11213b-523a-4725-9ac1-aba5987093b2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.620807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.621686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.622188 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.622613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] PATH_INFO: `/v3/projects/cc38c9208bfe4cda947eb8968e5bf152/users/71cc64589e43465c880ece8afcc03a17/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.624762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc38c9208bfe4cda947eb8968e5bf152, user_id=71cc64589e43465c880ece8afcc03a17, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.628203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.642176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1336/2658] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:44 2022] PUT /identity/v3/projects/cc38c9208bfe4cda947eb8968e5bf152/users/71cc64589e43465c880ece8afcc03a17/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:44.653489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3253b-0338-49db-87fd-027be665711e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.653789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3253b-0338-49db-87fd-027be665711e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.654066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3253b-0338-49db-87fd-027be665711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.702409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-ade3253b-0338-49db-87fd-027be665711e None None] MFA Rules not processed for user `71cc64589e43465c880ece8afcc03a17`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:44.703833 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ade3253b-0338-49db-87fd-027be665711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.720380 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.722127 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.794885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1323/2659] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:44.798137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71570595-0f55-4d9c-943e-e89b296aebc2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.806981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.807153 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3253b-0338-49db-87fd-027be665711e None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:44.807723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.808023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.808350 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] PATH_INFO: `/v3/projects/21ada22fb4d34a948854107aee99b180` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.808658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ade3253b-0338-49db-87fd-027be665711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.809562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ada22fb4d34a948854107aee99b180)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.811587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.859838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ada22fb4d34a948854107aee99b180', 'request_id': 'req-d637a278-627c-4f04-bfe4-482602767e4d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.866415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3253b-0338-49db-87fd-027be665711e None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:44.868365 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ada22fb4d34a948854107aee99b180', 'request_id': 'req-d637a278-627c-4f04-bfe4-482602767e4d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.874303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ada22fb4d34a948854107aee99b180', 'request_id': 'req-d637a278-627c-4f04-bfe4-482602767e4d'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.876078 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:44.887850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] The token cache is being invalidate because project 21ada22fb4d34a948854107aee99b180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:44.889304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.889546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d637a278-627c-4f04-bfe4-482602767e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:44.891266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1337/2660] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:44 2022] DELETE /identity/v3/projects/21ada22fb4d34a948854107aee99b180 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:44.896006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6dfe-d714-43c0-887a-39d0a753579a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.896294 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6dfe-d714-43c0-887a-39d0a753579a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.896566 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6dfe-d714-43c0-887a-39d0a753579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.916284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-740a6dfe-d714-43c0-887a-39d0a753579a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:44.917570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-740a6dfe-d714-43c0-887a-39d0a753579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.918566 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:44.919253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.919662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.919831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.921920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:44.951188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:44.976976 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1338/2661] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:44.981273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:44.982877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:44.985257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:44.985540 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:44.985813 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:44.992676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1324/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:45.005691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:45.006965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:45.015296 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1339/2663] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:45.019626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.030602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e961b8-ebff-4fdd-b232-ebc5ebb061a8 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.035473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.036085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.036367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.036753 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.037409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:45.037721 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.039407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.051102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1325/2664] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:45.055457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.065439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedeeb1a-1302-416a-ac2b-463b2d6966f2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.070401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.071046 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.071329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.071592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.072232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.074209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.119327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1340/2665] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:45.121794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.132177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0ddc-5f46-4add-92ed-c0596c13b214 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.192052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.192669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.192987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.193264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.193909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.195909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.219301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1326/2666] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:45.223849 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.246166 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cf8f3-0476-466b-b128-df9547ad5548 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.258191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.258775 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.259123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.259400 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.260064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.262658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.274578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1341/2667] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:45.278677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.291310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d429-334b-475f-8425-ba5130d24efe admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.297137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.297759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.298422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.298694 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.313312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.313312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61158192-d987-4e1b-a52f-07220b57a68d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.313312 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1327/2668] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:45.316988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.327457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420abb0c-9eb1-4cf6-b46c-79a70b58e2d3 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.332686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.333352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.333652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.333909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] PATH_INFO: `/v3/projects/4d85beb91e2945e0abffe3de7440adab/users/0e12834db2be4514b3b6e3de2b595d36/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.349695 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d85beb91e2945e0abffe3de7440adab, user_id=0e12834db2be4514b3b6e3de2b595d36, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.352396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.366979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1342/2669] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:45 2022] PUT /identity/v3/projects/4d85beb91e2945e0abffe3de7440adab/users/0e12834db2be4514b3b6e3de2b595d36/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:45.378375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.378664 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.378991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.406422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] MFA Rules not processed for user `0e12834db2be4514b3b6e3de2b595d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:45.407601 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:45.473515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1328/2670] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:45.478961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.555735 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cc1bb-fa5d-4a38-ab35-73de6c93a89b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.561700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.562342 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.562712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.563020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.563766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.565717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.595008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1343/2671] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:45.599230 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.610918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29316656-3da3-4c20-9c6d-64d7b2307025 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.632110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:45.669154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.669744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.670021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.670291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.670945 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.672861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.698745 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1329/2672] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:45.702781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.713147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeccbb9-830e-4a04-a33d-26c6015bf864 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.718136 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.718720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.719099 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.719454 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.719991 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.721572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.733248 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1344/2673] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:45.737271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.747621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707019ff-8627-450e-b84b-8ce298b01d59 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.752722 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.753379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.753661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.753918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] PATH_INFO: `/v3/projects/fc5cee00771d431c87a7c9d90587de4e/users/600ae783f3c940f48572c2fe7c9be028/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.769706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5cee00771d431c87a7c9d90587de4e, user_id=600ae783f3c940f48572c2fe7c9be028, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.772679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.787972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1330/2674] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:45 2022] PUT /identity/v3/projects/fc5cee00771d431c87a7c9d90587de4e/users/600ae783f3c940f48572c2fe7c9be028/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:45.791950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.803361 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884235-18d4-400f-9eb2-68b76bd31352 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.807065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:45.865514 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.866124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.866402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.866670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.867334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.868930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f45697-6e1e-4fe6-b492-3677b8135258 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.878729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1345/2675] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:45.883080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:45.893374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f0436-a7f5-469e-9a0a-b4d5b739fe1b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:45.898517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:45.899189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.899465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.899733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] PATH_INFO: `/v3/projects/fc5cee00771d431c87a7c9d90587de4e/users/600ae783f3c940f48572c2fe7c9be028/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.901764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5cee00771d431c87a7c9d90587de4e, user_id=600ae783f3c940f48572c2fe7c9be028, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:45.904442 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:45.915129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1331/2676] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:45 2022] PUT /identity/v3/projects/fc5cee00771d431c87a7c9d90587de4e/users/600ae783f3c940f48572c2fe7c9be028/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:45.925412 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:45.925582 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:45.925875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:45.953404 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] MFA Rules not processed for user `600ae783f3c940f48572c2fe7c9be028`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:45.954619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:46.023769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1346/2677] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:46.035209 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:46.036991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:46.095277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186178e9-edd5-430a-91ee-64a567cdcee1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:46.099477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:46.100087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:46.100365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:46.100635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:46.110787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:46.112272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:46.121757 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1332/2678] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:46.207555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:46.217882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181450-b579-4838-a80f-c2ae4d582d9b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:46.221928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:46.222523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:46.222794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:46.223094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:46.287366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:46.288842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3395005-efdd-4697-9e53-fcd39e22394c service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:46.297819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1347/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:48.642298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:48.644140 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:48.709662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055abbbe-26e8-42db-aebc-759eee9d72fc service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:48.714827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:48.715464 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:48.715748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:48.716014 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] PATH_INFO: `/v3/users/c9d344ee1daf4c9c8e9abeffd4b3ce45` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:48.717211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d344ee1daf4c9c8e9abeffd4b3ce45)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:48.719331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:48.741716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:48.769786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d344ee1daf4c9c8e9abeffd4b3ce45'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:48.770422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d344ee1daf4c9c8e9abeffd4b3ce45'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:48.781768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d344ee1daf4c9c8e9abeffd4b3ce45'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:48.788955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d344ee1daf4c9c8e9abeffd4b3ce45'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:48.789705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d344ee1daf4c9c8e9abeffd4b3ce45'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:48.803820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1333/2680] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:48 2022] DELETE /identity/v3/users/c9d344ee1daf4c9c8e9abeffd4b3ce45 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:48.808521 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:48.809034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:48.809139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:48.829536 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:48.830754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:48.852543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:48.892021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1348/2681] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:48.901966 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:48.903644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:48.986348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed72272-abf9-409b-81fe-5e83c4912b10 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:48.986348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:48.986348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:48.986348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:48.986742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:48.988007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:48.989429 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:49.000865 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1334/2682] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:49.312786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:49.314858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:49.346001 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3b2cf-74ea-471e-97c5-1930924589ad None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:49.356362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:49.357815 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:49.359044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:49.359667 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] PATH_INFO: `/v3/projects/2b3535b850f14e30b5cd40e1ba75f8e3` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:49.367002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3535b850f14e30b5cd40e1ba75f8e3)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:49.367002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:49.419756 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3535b850f14e30b5cd40e1ba75f8e3', 'request_id': 'req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.427218 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3535b850f14e30b5cd40e1ba75f8e3', 'request_id': 'req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.433573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3535b850f14e30b5cd40e1ba75f8e3', 'request_id': 'req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.444827 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] The token cache is being invalidate because project 2b3535b850f14e30b5cd40e1ba75f8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:49.445198 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.445899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.447802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1349/2683] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:49 2022] DELETE /identity/v3/projects/2b3535b850f14e30b5cd40e1ba75f8e3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:49.452252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:49.462330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8908ebc-3f3a-48d1-90c1-82e7afbf6154 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:49.525708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:49.526424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:49.526658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:49.527092 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] PATH_INFO: `/v3/users/71cc64589e43465c880ece8afcc03a17` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:49.528592 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cc64589e43465c880ece8afcc03a17)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:49.530659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:49.586324 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:49.589239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cc64589e43465c880ece8afcc03a17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.590001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cc64589e43465c880ece8afcc03a17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.604149 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cc64589e43465c880ece8afcc03a17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.611419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cc64589e43465c880ece8afcc03a17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.612300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cc64589e43465c880ece8afcc03a17'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:49.629070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1335/2684] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:49 2022] DELETE /identity/v3/users/71cc64589e43465c880ece8afcc03a17 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:49.919565 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:49.921068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:49.938144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cca8d-1c77-4e05-8a8d-655e1564e7bf admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:50.007082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:50.007742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:50.008017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:50.008279 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] PATH_INFO: `/v3/projects/cc38c9208bfe4cda947eb8968e5bf152` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:50.009411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc38c9208bfe4cda947eb8968e5bf152)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:50.011347 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:50.062863 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc38c9208bfe4cda947eb8968e5bf152', 'request_id': 'req-06f1b19b-ecbf-407e-bb15-1364a0ef5668'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:50.065383 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:50.071314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc38c9208bfe4cda947eb8968e5bf152', 'request_id': 'req-06f1b19b-ecbf-407e-bb15-1364a0ef5668'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:50.077829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc38c9208bfe4cda947eb8968e5bf152', 'request_id': 'req-06f1b19b-ecbf-407e-bb15-1364a0ef5668'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:50.089492 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] The token cache is being invalidate because project cc38c9208bfe4cda947eb8968e5bf152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:50.090290 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:50.090997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:50.092576 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1350/2685] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:49 2022] DELETE /identity/v3/projects/cc38c9208bfe4cda947eb8968e5bf152 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:53.359061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:53.361119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.375070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa727-df56-4368-8dee-d40af7fb2cc1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:53.436107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:53.436780 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.436922 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:53.437476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.437476 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] PATH_INFO: `/v3/users/0e12834db2be4514b3b6e3de2b595d36` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.438441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.438576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e12834db2be4514b3b6e3de2b595d36)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:53.440497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:53.451329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1b19b-ecbf-407e-bb15-1364a0ef5668 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:53.457239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:53.458636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.458963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.459247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] PATH_INFO: `/v3/users/bffcbbc966f94d15a034241239ea1899` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.460417 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffcbbc966f94d15a034241239ea1899)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:53.462372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:53.491077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e12834db2be4514b3b6e3de2b595d36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.491765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e12834db2be4514b3b6e3de2b595d36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.505225 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e12834db2be4514b3b6e3de2b595d36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.512264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e12834db2be4514b3b6e3de2b595d36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.512943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-269cf201-cf1f-4023-94e8-03216904069f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e12834db2be4514b3b6e3de2b595d36'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.514463 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffcbbc966f94d15a034241239ea1899'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.528204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1336/2686] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:53 2022] DELETE /identity/v3/users/0e12834db2be4514b3b6e3de2b595d36 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:53.529945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffcbbc966f94d15a034241239ea1899'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.532710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.532994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.533284 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.538131 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffcbbc966f94d15a034241239ea1899'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.538887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffcbbc966f94d15a034241239ea1899'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.539828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-567c1fac-2c14-415c-94cf-1873f1bf7a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffcbbc966f94d15a034241239ea1899'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:53.554337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1351/2687] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:53 2022] DELETE /identity/v3/users/bffcbbc966f94d15a034241239ea1899 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:53.558067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:53.558947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.559306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.559426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.559584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.579132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:53.580456 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.593624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:53.622039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1337/2688] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:53.632064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:53.633641 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.645441 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1352/2689] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:56:53.655525 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:53.656998 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:53.696245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921ea07-6a4e-49bb-b0fd-3076cec0dc30 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:53.700159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:53.700778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.701058 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.701343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.711454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:53.712931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:53.726892 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad550a-904f-4278-9e99-d17b644c8e2e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:53.727040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1338/2690] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:53.739616 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:53.740308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:53.740592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:53.740871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:53.749810 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:53.753187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:53.755774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:53.773768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1353/2691] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:56:54.071755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:54.073236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:54.087996 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f04b60-41f0-4862-bc2d-fce707d36aac service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:54.093713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:54.094365 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.094644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.095008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] PATH_INFO: `/v3/projects/4d85beb91e2945e0abffe3de7440adab` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:54.096141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d85beb91e2945e0abffe3de7440adab)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:54.098080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:54.135202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:54.137379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:54.152511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b71e-c01d-4f44-849e-d648d4662b49 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:54.153421 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d85beb91e2945e0abffe3de7440adab', 'request_id': 'req-1ec90339-899d-409e-bafe-7387cbff60e7'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.162682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d85beb91e2945e0abffe3de7440adab', 'request_id': 'req-1ec90339-899d-409e-bafe-7387cbff60e7'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.169357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d85beb91e2945e0abffe3de7440adab', 'request_id': 'req-1ec90339-899d-409e-bafe-7387cbff60e7'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.181576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] The token cache is being invalidate because project 4d85beb91e2945e0abffe3de7440adab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:54.181864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.182512 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.184004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1339/2692] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:54 2022] DELETE /identity/v3/projects/4d85beb91e2945e0abffe3de7440adab => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:54.188452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:54.189624 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:54.214891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:54.215468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.215761 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.216023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] PATH_INFO: `/v3/projects/b596031cb0e84c3a84c725588e9d3fc8` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:54.217191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b596031cb0e84c3a84c725588e9d3fc8)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:54.219125 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:54.260204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec90339-899d-409e-bafe-7387cbff60e7 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:54.269210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:54.271489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b596031cb0e84c3a84c725588e9d3fc8', 'request_id': 'req-22d6359f-74d0-402f-aa07-1d67896fc293'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.280337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b596031cb0e84c3a84c725588e9d3fc8', 'request_id': 'req-22d6359f-74d0-402f-aa07-1d67896fc293'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.287997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b596031cb0e84c3a84c725588e9d3fc8', 'request_id': 'req-22d6359f-74d0-402f-aa07-1d67896fc293'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.305023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] The token cache is being invalidate because project b596031cb0e84c3a84c725588e9d3fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:54.305489 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.306444 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.308585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1354/2693] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:54 2022] DELETE /identity/v3/projects/b596031cb0e84c3a84c725588e9d3fc8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:54.327097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:54.327770 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.328206 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.328494 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] PATH_INFO: `/v3/users/600ae783f3c940f48572c2fe7c9be028` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:54.329806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=600ae783f3c940f48572c2fe7c9be028)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:54.331980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:54.387567 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600ae783f3c940f48572c2fe7c9be028'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.388373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600ae783f3c940f48572c2fe7c9be028'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.402258 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600ae783f3c940f48572c2fe7c9be028'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.409657 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600ae783f3c940f48572c2fe7c9be028'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.410749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-27902bae-a7f0-463c-a7be-8743c12a0fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600ae783f3c940f48572c2fe7c9be028'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.427496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1340/2694] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:56:54 2022] DELETE /identity/v3/users/600ae783f3c940f48572c2fe7c9be028 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:56:54.734325 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:54.735959 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:54.749170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6359f-74d0-402f-aa07-1d67896fc293 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:54.808028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:54.808596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.808875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.809148 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] PATH_INFO: `/v3/projects/fc5cee00771d431c87a7c9d90587de4e` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:54.810284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5cee00771d431c87a7c9d90587de4e)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:54.812241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:54.847591 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:54.868117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5cee00771d431c87a7c9d90587de4e', 'request_id': 'req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.877137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5cee00771d431c87a7c9d90587de4e', 'request_id': 'req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.882907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5cee00771d431c87a7c9d90587de4e', 'request_id': 'req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.893856 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] The token cache is being invalidate because project fc5cee00771d431c87a7c9d90587de4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:56:54.894139 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.894772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-4fe5b3ba-9d1a-4e11-9174-4adbc6892543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:56:54.896289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1355/2695] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:56:54 2022] DELETE /identity/v3/projects/fc5cee00771d431c87a7c9d90587de4e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:54.901270 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.901551 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.901821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:54.921748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:54.923054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:54.986794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1341/2696] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:54.995968 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:54.996274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:54.996545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.016640 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:55.017838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:55.026256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1356/2697] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:55.030502 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.040965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b8dc-e430-4b64-ab31-b655cb502b3a None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.046098 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.046696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.047008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.047273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.047938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:56:55.048238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.049845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.062308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1342/2698] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:56:55.066622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.076282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9e86-0a07-47ec-bded-0cfde1833f5e None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.081993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.082597 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.082919 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.083194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.083830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.085773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.115714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1357/2699] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:55 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:55.120417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.133060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57bec2-1062-4d9c-bc12-4fbcf116b9b0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.152377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:55.195395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.196000 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.196277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.196545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.197188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.199158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.222257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1343/2700] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:55 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:55.226697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.237962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15f77f-67a2-4a3c-a2be-a3ab6c7bed18 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.243253 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.243854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.244146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.244407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.245050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.246650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.257064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1358/2701] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:55.261459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.272211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef13f97-7060-4a4b-b557-cbe199afeff3 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.278005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.278850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.279238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.279505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.280179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.281779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc569f1c-cc75-46f0-89f2-9d77e79798dd admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.293407 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1344/2702] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:55.298234 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.313872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e574e5f-e1ee-4eec-b423-8f4037770629 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.321857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.322778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.323110 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.323438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] PATH_INFO: `/v3/projects/ba2b420e41384d00b5ade6a7f753efc5/users/25e428b1a0d246dfa98e566e94322456/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.340132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2b420e41384d00b5ade6a7f753efc5, user_id=25e428b1a0d246dfa98e566e94322456, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.342769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.357757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1359/2703] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:55 2022] PUT /identity/v3/projects/ba2b420e41384d00b5ade6a7f753efc5/users/25e428b1a0d246dfa98e566e94322456/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:55.362767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771febc0-3d8e-44c4-9c5d-828836ef3928 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.363103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771febc0-3d8e-44c4-9c5d-828836ef3928 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.363394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771febc0-3d8e-44c4-9c5d-828836ef3928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.382396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-771febc0-3d8e-44c4-9c5d-828836ef3928 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:55.383617 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-771febc0-3d8e-44c4-9c5d-828836ef3928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:55.448383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1345/2704] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:55.458394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:55.460329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:55.517357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777927-9d14-489b-a50f-186cfedafb8e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:55.521242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:55.521893 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:55.522158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:55.522442 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:55.531961 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:55.533503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:55.548123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1360/2705] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:56.205063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.205400 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.205700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.240998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] MFA Rules not processed for user `25e428b1a0d246dfa98e566e94322456`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:56.242265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:56.317237 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1346/2706] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:56.321865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.333207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58880-6c14-4314-86d9-cac2f3aa9cc1 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.337826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.338420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.338699 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.339009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.339679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.341588 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.371319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1361/2707] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:56:56.375457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.386892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4be9-c6df-41be-8530-2b413b9481b6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.412667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:56.443085 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.443699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.443982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.444243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.444904 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.446820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.467674 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1347/2708] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:56:56 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:56:56.471902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.482814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa958c-bb37-4bf6-8392-35cf846d2700 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.487993 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.488599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.488909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.489181 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.489821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.491455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.502164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1362/2709] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:56.506249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.517267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b484bc-6552-482b-beda-dd5fc625737e admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.523813 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.524459 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.524737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.524999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] PATH_INFO: `/v3/projects/c058036a4e2f412e9baf498b9db02a6a/users/725941de3aec459da51bf7e9445c66e6/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.541250 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c058036a4e2f412e9baf498b9db02a6a, user_id=725941de3aec459da51bf7e9445c66e6, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.543950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.558880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1348/2710] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:56 2022] PUT /identity/v3/projects/c058036a4e2f412e9baf498b9db02a6a/users/725941de3aec459da51bf7e9445c66e6/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:56.563679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.574097 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1b1f0-6221-4d02-b5e8-a79fd3df8359 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.592384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:56:56.632697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.633237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.633523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.633797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.634433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.636083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f21db4-2ead-49ee-8ffd-610ea9d27a26 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.649351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1363/2711] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:56:56.654762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:56.667620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b0ce0-c3a4-4bfd-8181-b6e37de17e24 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:56.675385 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:56.676039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:56.676325 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:56.676572 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] PATH_INFO: `/v3/projects/c058036a4e2f412e9baf498b9db02a6a/users/725941de3aec459da51bf7e9445c66e6/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:56.678821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c058036a4e2f412e9baf498b9db02a6a, user_id=725941de3aec459da51bf7e9445c66e6, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:56.681509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:56.693316 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1349/2712] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:56:56 2022] PUT /identity/v3/projects/c058036a4e2f412e9baf498b9db02a6a/users/725941de3aec459da51bf7e9445c66e6/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:56:57.343888 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:57.344419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:57.344419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:57.373564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] MFA Rules not processed for user `725941de3aec459da51bf7e9445c66e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:56:57.374826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:57.446528 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1364/2713] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:56:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:56:57.458166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:57.459834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:56:57.517162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b3d32-33bd-4946-ab0c-958f724d0a8d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:57.522013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:57.522821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:57.523147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:57.523503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:57.587517 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:57.589909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:57.603549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1350/2714] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:56:58.985960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:56:59.004040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d035b-0dfd-43ad-ae0e-414be1fc793a None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:56:59.010324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:56:59.011247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:56:59.011562 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:56:59.011923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:56:59.024950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:56:59.026391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ae341-7876-4876-b7e6-64e6e9d6ec82 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:56:59.036653 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1365/2715] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:56:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:01.889199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:01.890798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:01.933514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:01.960882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef8a-3b6c-4b43-a916-840b6beda57c service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:01.965548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:01.966178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:01.966451 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:01.966717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] PATH_INFO: `/v3/users/e7363c6e316047b787e2befca93cd28e` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:01.967897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7363c6e316047b787e2befca93cd28e)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:01.969831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:02.016552 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7363c6e316047b787e2befca93cd28e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.017190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7363c6e316047b787e2befca93cd28e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.029926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7363c6e316047b787e2befca93cd28e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.037696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7363c6e316047b787e2befca93cd28e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.038439 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7363c6e316047b787e2befca93cd28e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.053875 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1351/2716] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:01 2022] DELETE /identity/v3/users/e7363c6e316047b787e2befca93cd28e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:02.058592 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:02.059095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:02.059220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:02.078486 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:02.079707 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:02.143026 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1366/2717] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:02.153030 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:02.154669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:02.212307 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023484-6997-4417-9da5-c2be94b250de admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:02.216426 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:02.217276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:02.217558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:02.217829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:02.227376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:02.228801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:02.240662 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1352/2718] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:02.521216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:02.523440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:02.540872 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444a34c-3505-47ba-8f16-647324e79d95 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:02.546098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:02.546918 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:02.547202 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:02.547474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] PATH_INFO: `/v3/projects/154f93861c3546eab982f68ea66f158c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:02.548628 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154f93861c3546eab982f68ea66f158c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:02.550568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:02.584256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:02.606966 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154f93861c3546eab982f68ea66f158c', 'request_id': 'req-2e977e9d-7977-4f94-b61e-4d6490cf4480'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.619337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154f93861c3546eab982f68ea66f158c', 'request_id': 'req-2e977e9d-7977-4f94-b61e-4d6490cf4480'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.625885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154f93861c3546eab982f68ea66f158c', 'request_id': 'req-2e977e9d-7977-4f94-b61e-4d6490cf4480'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.637848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] The token cache is being invalidate because project 154f93861c3546eab982f68ea66f158c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:02.638138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.638952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.640532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1367/2719] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:02 2022] DELETE /identity/v3/projects/154f93861c3546eab982f68ea66f158c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:02.645433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:02.659810 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81073042-54df-4b20-a3b9-9d20a7ec2b0a service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:02.711911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:02.719558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:02.720210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:02.720489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:02.720765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] PATH_INFO: `/v3/users/5b5e777716034362917cedc511391973` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:02.722145 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5e777716034362917cedc511391973)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:02.724364 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:02.771620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5e777716034362917cedc511391973'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.772273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5e777716034362917cedc511391973'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.783968 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5e777716034362917cedc511391973'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.791269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5e777716034362917cedc511391973'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.791897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-df59577a-8e84-4370-a299-dd922f4a89e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5e777716034362917cedc511391973'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:02.807690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1353/2720] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:02 2022] DELETE /identity/v3/users/5b5e777716034362917cedc511391973 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:03.092720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.094358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:03.110762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e977e9d-7977-4f94-b61e-4d6490cf4480 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.189948 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.190697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.196620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.197129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] PATH_INFO: `/v3/projects/2542c4fc300b41068daa3e174a16d4eb` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.198746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2542c4fc300b41068daa3e174a16d4eb)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.210785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.259882 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2542c4fc300b41068daa3e174a16d4eb', 'request_id': 'req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:03.267680 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2542c4fc300b41068daa3e174a16d4eb', 'request_id': 'req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:03.273556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2542c4fc300b41068daa3e174a16d4eb', 'request_id': 'req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:03.284982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] The token cache is being invalidate because project 2542c4fc300b41068daa3e174a16d4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:03.285266 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:03.285932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-80016a4a-63bf-48ee-a6f5-76b5bcb075c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:03.287450 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1368/2721] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:03 2022] DELETE /identity/v3/projects/2542c4fc300b41068daa3e174a16d4eb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:03.292929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.293412 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.293532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.312926 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:03.314135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:03.377288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1354/2722] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:03.386340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.386625 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.386945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.408197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:03.409420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:03.418889 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1369/2723] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:03.423379 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.434001 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c344cf4-445f-4363-a844-ec683e3d78e5 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.439200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.439868 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.440114 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.440418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.441066 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:03.441373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.443064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.455987 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1355/2724] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:03.460771 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.474123 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c9122-1b02-4cf4-81c9-e21ab77442a9 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.483019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.483901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.484392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.484715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.485607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.488174 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.516543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1370/2725] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:03.520953 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.531880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff631d83-de8e-4f27-a3e7-771d65110bd1 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.592629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.593501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.593915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.594281 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.595342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.598450 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.613998 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:03.629087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1356/2726] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:03.633413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.644666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d98c4-73d0-454d-a66e-27745b5440a4 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.649304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.649895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.650164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.650433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.651118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.652698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.662712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1371/2727] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:03.666676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.677332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0707cb9-f9db-4c02-9d0f-52d5ffc8d6f2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.683932 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.684534 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.684797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.685065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.685846 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.687545 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf1ecc-c099-4543-8a3a-09dbace1d902 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.701646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1357/2728] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:03.706270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.712909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:03.716805 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5040ca-491c-43a6-8118-b9d1fed79427 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.722709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.723511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.723982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.724457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] PATH_INFO: `/v3/projects/7177b6441b8b4fedbea7eaac6bb80eca/users/83373b5e6e424964bf42f1387a4d9d49/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.739468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7177b6441b8b4fedbea7eaac6bb80eca, user_id=83373b5e6e424964bf42f1387a4d9d49, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.742113 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.756923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1372/2729] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:03 2022] PUT /identity/v3/projects/7177b6441b8b4fedbea7eaac6bb80eca/users/83373b5e6e424964bf42f1387a4d9d49/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:03.767484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.767764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.768031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.796766 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] MFA Rules not processed for user `83373b5e6e424964bf42f1387a4d9d49`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:03.798041 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:03.864921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1358/2730] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:03.871730 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.939134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b209-9fdc-4edc-bd65-16ec5599b5cd admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:03.944584 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:03.945199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:03.945506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:03.945786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:03.946424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:03.948366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:03.981649 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1373/2731] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:03.986121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:03.997830 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54135b-f94e-4dc2-a1db-3eff7aaa7ef8 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.076835 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.077478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.077773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.078037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.078700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.080760 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.103827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1359/2732] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:03 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:04.108495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.119637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81b0c-189c-452b-bb66-bd2b0fc947ec admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.124929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.125533 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.125808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.126074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.126710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.128381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.139453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1374/2733] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:04.144183 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.155052 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a757c-4627-429e-b52e-10e9374e98b6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.160329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.161015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.161297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.161570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] PATH_INFO: `/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2/users/323374f6545e4ec49865d05934ed6506/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.177510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b037e5b9b414e56a8dbf4df8ce061e2, user_id=323374f6545e4ec49865d05934ed6506, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.180249 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.206743 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1360/2734] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:04 2022] PUT /identity/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2/users/323374f6545e4ec49865d05934ed6506/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:04.210760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.221580 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93fe-a01a-4816-be51-299c4e469571 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.326430 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.327154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.327504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.327842 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.328551 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.330230 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.339979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1375/2735] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:04.344318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.377993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:04.397292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812839-75e9-4ddc-a78f-2dc5bc3989e0 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.401994 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.402167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.402682 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.402984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.403252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] PATH_INFO: `/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2/users/323374f6545e4ec49865d05934ed6506/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.403824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:04.405393 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b037e5b9b414e56a8dbf4df8ce061e2, user_id=323374f6545e4ec49865d05934ed6506, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.408288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce62c-9199-4867-a76e-a69ff60dfdbc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.417851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:04.419321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1361/2736] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:04 2022] PUT /identity/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2/users/323374f6545e4ec49865d05934ed6506/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:04.430165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.430503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.430732 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.466863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] MFA Rules not processed for user `323374f6545e4ec49865d05934ed6506`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:04.468067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:04.485949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896a15-2541-4a4c-a915-eb87d267401c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.555158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1362/2737] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:04.564488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.565241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.565517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.565804 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.567603 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:04.567854 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.570100 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:04.630390 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.632038 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.632181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7af24-d842-4d1a-aea9-160492e15c27 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.635913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.636541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.636902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.637240 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.642063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1376/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:04.647972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.649436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.659658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1363/2739] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:57:04.708205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:04.709733 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:04.721620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e09d-cdaa-49d1-8dcf-4431a6ba3882 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:04.725303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:04.725933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:04.726208 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:04.726474 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:04.784654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:04.788811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:04.790666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd73d1d-7da8-40f0-997c-59f1f568979a service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:04.800690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1377/2740] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:57:08.222377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:08.343192 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bf3bf-5f6b-4075-8494-8403fb81fab4 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:08.349408 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:08.350188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:08.350467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:08.350735 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] PATH_INFO: `/v3/users/83373b5e6e424964bf42f1387a4d9d49` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:08.351899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83373b5e6e424964bf42f1387a4d9d49)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:08.353850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:08.366626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:08.414151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83373b5e6e424964bf42f1387a4d9d49'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:08.414863 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83373b5e6e424964bf42f1387a4d9d49'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:08.430701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83373b5e6e424964bf42f1387a4d9d49'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:08.439655 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83373b5e6e424964bf42f1387a4d9d49'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:08.440411 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83373b5e6e424964bf42f1387a4d9d49'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:08.455409 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1364/2741] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:08 2022] DELETE /identity/v3/users/83373b5e6e424964bf42f1387a4d9d49 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:08.460122 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:08.460744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:08.460871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:08.491359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:08.492700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:08.598411 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1378/2742] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:57:08.611661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:08.615883 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:08.679068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d004177-45bd-4183-8d5f-a60e77933865 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:08.682581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:08.683233 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:08.683510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:08.683776 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:08.694032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:08.695531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:08.705339 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1365/2743] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:08.984215 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:08.985689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.000964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8477c95-b09e-496c-9363-7e14013b39a4 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.013115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.013785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.014063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.014329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] PATH_INFO: `/v3/projects/7177b6441b8b4fedbea7eaac6bb80eca` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.015880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7177b6441b8b4fedbea7eaac6bb80eca)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.017837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:09.068374 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7177b6441b8b4fedbea7eaac6bb80eca', 'request_id': 'req-eb620c5f-cb9c-46ad-9236-e19ff058922a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.076212 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7177b6441b8b4fedbea7eaac6bb80eca', 'request_id': 'req-eb620c5f-cb9c-46ad-9236-e19ff058922a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.081828 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7177b6441b8b4fedbea7eaac6bb80eca', 'request_id': 'req-eb620c5f-cb9c-46ad-9236-e19ff058922a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.092956 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] The token cache is being invalidate because project 7177b6441b8b4fedbea7eaac6bb80eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:09.093247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.093990 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.095654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1379/2744] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:08 2022] DELETE /identity/v3/projects/7177b6441b8b4fedbea7eaac6bb80eca => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:09.099918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:09.110272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb69f7-f8e8-474a-ab7e-970f623e0cce service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.169910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.170539 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.170850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.171111 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] PATH_INFO: `/v3/users/323374f6545e4ec49865d05934ed6506` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.172268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=323374f6545e4ec49865d05934ed6506)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.174222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:09.209410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:09.223186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '323374f6545e4ec49865d05934ed6506'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.223801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '323374f6545e4ec49865d05934ed6506'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.236625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '323374f6545e4ec49865d05934ed6506'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.244277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '323374f6545e4ec49865d05934ed6506'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.244995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '323374f6545e4ec49865d05934ed6506'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.259699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1366/2745] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:09 2022] DELETE /identity/v3/users/323374f6545e4ec49865d05934ed6506 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:09.543727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:09.546237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.563869 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb620c5f-cb9c-46ad-9236-e19ff058922a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.589962 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:09.592047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.644650 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.645510 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.645821 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.646095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] PATH_INFO: `/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.647355 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b037e5b9b414e56a8dbf4df8ce061e2)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.649708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:09.658591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6feb-f21d-4f59-b2e8-6378e7562f76 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.662318 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.662940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.663222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.663496 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.665402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.665894 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:09.681969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.684223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087ae5f-0c1d-43cc-a83a-e2a103b77466 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:09.699042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1367/2746] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 03:57:09.713104 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b037e5b9b414e56a8dbf4df8ce061e2', 'request_id': 'req-9ef392d8-2003-43f8-acae-0a8c5342ca1b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.723269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b037e5b9b414e56a8dbf4df8ce061e2', 'request_id': 'req-9ef392d8-2003-43f8-acae-0a8c5342ca1b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.736512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b037e5b9b414e56a8dbf4df8ce061e2', 'request_id': 'req-9ef392d8-2003-43f8-acae-0a8c5342ca1b'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.749152 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] The token cache is being invalidate because project 4b037e5b9b414e56a8dbf4df8ce061e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:09.749440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.750515 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9ef392d8-2003-43f8-acae-0a8c5342ca1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:09.752370 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1380/2747] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:09 2022] DELETE /identity/v3/projects/4b037e5b9b414e56a8dbf4df8ce061e2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:09.757264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.757799 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.758226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.813554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:09.816108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.886991 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1368/2748] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:09.895163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.895564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.895702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.916269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:09.917466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:09.929210 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1381/2749] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:09.930666 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:09.940804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60cf3b-7800-4030-8c6c-56e51ff81872 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.946200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.946446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.946706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.947005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.947902 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:09.948043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.949972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:09.963594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1369/2750] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:09.967808 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:09.977429 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036caa56-cdaf-4b62-88d1-cb70a37a8edc None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:09.982633 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:09.983158 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:09.983469 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:09.983705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:09.984316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:09.986213 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.019663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1382/2751] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:09 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:10.023811 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.033916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc851-501e-4401-9db5-7b09b0ab1741 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.049231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:10.103351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.104119 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.104448 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.104663 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.106043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.108038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.135271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1370/2752] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:10 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:10.139552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.160509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4e96a-51ed-496c-b310-affbe74ff8a8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.168951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.169678 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.170014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.170389 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.171149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.172850 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.198611 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1383/2753] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:10.199804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.212399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56535-d966-4679-810d-e1c84a0da1b8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.217786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.218399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.218676 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.219154 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.219807 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.222063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f474-8cf2-4438-906f-83b627f3d317 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.238820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1371/2754] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:10.240042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.251112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f2d-a245-41ed-9fd7-69b4f11cce82 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.263997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.265639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.266169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.266622 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] PATH_INFO: `/v3/projects/81fec515e17d412398ade6c10c71fa72/users/dbacebadaf0249228e439c926a45e103/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.289806 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81fec515e17d412398ade6c10c71fa72, user_id=dbacebadaf0249228e439c926a45e103, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.292979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.319433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1384/2755] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:10 2022] PUT /identity/v3/projects/81fec515e17d412398ade6c10c71fa72/users/dbacebadaf0249228e439c926a45e103/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:10.330088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.330474 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.330829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.360063 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] MFA Rules not processed for user `dbacebadaf0249228e439c926a45e103`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:10.361560 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:10.437640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1372/2756] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:10.442878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.510573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db0fc5-bfa8-421c-b095-f3f370bd3f42 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.515309 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.515931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.516207 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.516477 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.517107 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.519142 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.546874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1385/2757] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:10 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:10.551820 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.563688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285680b-be95-447e-aa96-9158fe6ec08c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.624503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.625137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.625403 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.625672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.626301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.628340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.656955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1373/2758] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:10 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:10.660531 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.672609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9529-20be-4b41-8464-f3463b16128b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.677839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.678472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.678765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.679075 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.679790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.681453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.693054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1386/2759] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:10.699185 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.711831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158a8b8-2b0f-49b4-b340-062f0fbe5c31 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.716989 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.717669 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.717949 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.718224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] PATH_INFO: `/v3/projects/1e32d292f23d48e48185e99897aa1f25/users/834e7db0d8a4438e81a35b148cad4954/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.731532 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:10.736519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e32d292f23d48e48185e99897aa1f25, user_id=834e7db0d8a4438e81a35b148cad4954, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.740935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.757911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1374/2760] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:10 2022] PUT /identity/v3/projects/1e32d292f23d48e48185e99897aa1f25/users/834e7db0d8a4438e81a35b148cad4954/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:10.762196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.773672 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c125be-e323-4a39-8ead-aa673919341e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.784388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:10.856866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.857495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.857811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.858079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.858729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.860366 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012abcab-2ecd-4ce6-b8a7-5714e443672c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.871226 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1387/2761] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:10.875497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:10.886142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684c5a-91f1-4bc7-b7e2-570d0594be07 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:10.891146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:10.891805 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.892080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.892346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] PATH_INFO: `/v3/projects/1e32d292f23d48e48185e99897aa1f25/users/834e7db0d8a4438e81a35b148cad4954/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.894420 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e32d292f23d48e48185e99897aa1f25, user_id=834e7db0d8a4438e81a35b148cad4954, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:10.897086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:10.914900 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1375/2762] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:10 2022] PUT /identity/v3/projects/1e32d292f23d48e48185e99897aa1f25/users/834e7db0d8a4438e81a35b148cad4954/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:10.919686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:10.919983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:10.920259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:10.949755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] MFA Rules not processed for user `834e7db0d8a4438e81a35b148cad4954`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:10.950988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:11.028556 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1388/2763] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:10 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:11.039486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:11.041608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:11.114015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1663-9c99-469b-980b-0723bf2328d6 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:11.120971 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:11.121620 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:11.121898 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:11.122171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:11.132190 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:11.133640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:11.144311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1376/2764] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:11.192895 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:11.194503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:11.208613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc02ee5-fc11-42bf-be2c-efcbba94d846 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:11.212739 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:11.213399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:11.213681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:11.213952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:11.280109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:11.281689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:11.291824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1389/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:12.746541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:12.750473 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:12.830830 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:12.832596 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:12.929835 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=100 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:13.006790 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969dd6-97db-49b7-9cec-f1ffaa8c8508 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:13.010096 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f4615-5346-44e3-9326-692661657d10 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:13.013967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:13.014930 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:13.015204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:13.015478 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] PATH_INFO: `/v3/users/25e428b1a0d246dfa98e566e94322456` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:13.018259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e428b1a0d246dfa98e566e94322456)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:13.019169 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:13.021094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:13.021390 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:13.021788 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:13.023196 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:13.029579 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:13.056171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:13.059044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:13.074452 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1377/2766] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:13.153944 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e428b1a0d246dfa98e566e94322456'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.177098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e428b1a0d246dfa98e566e94322456'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.187177 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e428b1a0d246dfa98e566e94322456'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.188112 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e428b1a0d246dfa98e566e94322456'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.188780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e428b1a0d246dfa98e566e94322456'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.232114 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1390/2767] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:12 2022] DELETE /identity/v3/users/25e428b1a0d246dfa98e566e94322456 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:13.489288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:13.503803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef572-6a15-4841-9154-58b4a26fecd6 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:13.569908 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:13.585208 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:13.586906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:13.586906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:13.586906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] PATH_INFO: `/v3/projects/ba2b420e41384d00b5ade6a7f753efc5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:13.587435 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2b420e41384d00b5ade6a7f753efc5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:13.589538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:13.651911 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2b420e41384d00b5ade6a7f753efc5', 'request_id': 'req-160884dc-2f12-40f5-85e6-ed74f708c61b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.662203 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2b420e41384d00b5ade6a7f753efc5', 'request_id': 'req-160884dc-2f12-40f5-85e6-ed74f708c61b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.668915 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2b420e41384d00b5ade6a7f753efc5', 'request_id': 'req-160884dc-2f12-40f5-85e6-ed74f708c61b'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.681850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] The token cache is being invalidate because project ba2b420e41384d00b5ade6a7f753efc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:13.682301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.682919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.688064 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1378/2768] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:13 2022] DELETE /identity/v3/projects/ba2b420e41384d00b5ade6a7f753efc5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:13.689797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:13.691736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:13.713176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c56fb-eaab-40de-aa13-845573718f72 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:13.784259 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:13.785619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:13.785901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:13.786222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] PATH_INFO: `/v3/users/725941de3aec459da51bf7e9445c66e6` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:13.787782 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725941de3aec459da51bf7e9445c66e6)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:13.795293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:13.852787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725941de3aec459da51bf7e9445c66e6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.859283 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:13.869788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725941de3aec459da51bf7e9445c66e6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.877837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725941de3aec459da51bf7e9445c66e6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.879535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725941de3aec459da51bf7e9445c66e6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.880402 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-876ea7ad-b647-4255-ac10-c46e8e564b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725941de3aec459da51bf7e9445c66e6'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:13.896118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1391/2769] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:13 2022] DELETE /identity/v3/users/725941de3aec459da51bf7e9445c66e6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:14.097199 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.099013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:14.116999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160884dc-2f12-40f5-85e6-ed74f708c61b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.203117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.203769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.204047 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.204471 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] PATH_INFO: `/v3/projects/c058036a4e2f412e9baf498b9db02a6a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.206108 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c058036a4e2f412e9baf498b9db02a6a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.208395 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.270186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c058036a4e2f412e9baf498b9db02a6a', 'request_id': 'req-78870220-9762-483c-89fe-1a5ae2799b39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:14.278874 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c058036a4e2f412e9baf498b9db02a6a', 'request_id': 'req-78870220-9762-483c-89fe-1a5ae2799b39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:14.284819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c058036a4e2f412e9baf498b9db02a6a', 'request_id': 'req-78870220-9762-483c-89fe-1a5ae2799b39'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:14.295933 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] The token cache is being invalidate because project c058036a4e2f412e9baf498b9db02a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:14.296347 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:14.297102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-78870220-9762-483c-89fe-1a5ae2799b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:14.298740 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1379/2770] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:14 2022] DELETE /identity/v3/projects/c058036a4e2f412e9baf498b9db02a6a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:14.303818 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648571a-447f-4117-8342-f466811f8caf None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.304106 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648571a-447f-4117-8342-f466811f8caf None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.304399 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648571a-447f-4117-8342-f466811f8caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.325462 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-b648571a-447f-4117-8342-f466811f8caf None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:14.326715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b648571a-447f-4117-8342-f466811f8caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:14.389780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1392/2771] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:57:14.398040 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.398335 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.398671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.421877 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:14.423625 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:14.435171 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1380/2772] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:14.436772 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648571a-447f-4117-8342-f466811f8caf None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.451312 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648571a-447f-4117-8342-f466811f8caf None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.456577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.457285 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.457629 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.457960 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.458696 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:14.459084 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.460792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.477720 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1393/2773] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:57:14.479260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.484211 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:14.497748 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da945-2b31-492d-9cd4-5c68c9b2fe16 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.504159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.504796 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.505078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.505349 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.506019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.508166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.539528 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1381/2774] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:14 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:14.541087 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.551524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a6c8f-eb5f-48b6-8737-bdeee4db16b0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.612014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.612711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.612840 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.613054 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.613686 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.615643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.638008 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1394/2775] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:14 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:14.642117 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.652576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e217d28-f137-4c5b-a5c6-8d544003e091 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.657388 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.658223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.658519 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.658798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.659501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.661200 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.674758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1382/2776] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:14.676176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.687329 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91019f6-48a3-4e58-8052-fcfd2fde3c45 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.692767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.693428 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.693741 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.694111 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.694768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.696568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffb06a-85a2-49db-83e1-49ecf58b7bdc admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.707690 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1395/2777] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:14.711897 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.722003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e536d-6ae4-4c0c-bcfe-baf28158f9ba admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.727132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.727821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.728091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.728361 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] PATH_INFO: `/v3/projects/e06555bf54b2487cb91ebf22a987d416/users/afb9eb7271564fd48a88cb94fd0b7991/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.745777 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06555bf54b2487cb91ebf22a987d416, user_id=afb9eb7271564fd48a88cb94fd0b7991, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.748695 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.762054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1383/2778] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:14 2022] PUT /identity/v3/projects/e06555bf54b2487cb91ebf22a987d416/users/afb9eb7271564fd48a88cb94fd0b7991/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:14.772485 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.772884 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.773372 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.809750 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] MFA Rules not processed for user `afb9eb7271564fd48a88cb94fd0b7991`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:14.811398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:14.844252 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:14.892134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1396/2779] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 03:57:14.905621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:14.908007 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:14.972300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fcc7-07dc-4a05-9396-8bf9cde03d01 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:14.976181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:14.976855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:14.977221 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:14.977531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:14.987243 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:14.988864 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d175c-c103-469a-b3bb-5c6f29aa60b6 service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:14.997856 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1384/2780] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:15.229938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:15.232823 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:15.318787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0c15a-63fa-4b69-a2d0-52770159f5f7 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:15.324203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:15.324979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:15.325426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:15.326039 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] PATH_INFO: `/v3/users/dbacebadaf0249228e439c926a45e103` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:15.327391 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbacebadaf0249228e439c926a45e103)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:15.329545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:15.412675 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbacebadaf0249228e439c926a45e103'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:15.428776 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbacebadaf0249228e439c926a45e103'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:15.437563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbacebadaf0249228e439c926a45e103'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:15.438396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbacebadaf0249228e439c926a45e103'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:15.439094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbacebadaf0249228e439c926a45e103'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:15.458512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1397/2781] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:15 2022] DELETE /identity/v3/users/dbacebadaf0249228e439c926a45e103 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:15.464276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:15.465143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:15.465416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:15.511553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:15.515750 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:15.621969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1385/2782] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:15.631784 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:15.633676 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:15.688134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:15.713257 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dca-a9ad-4970-9647-65dc210baa7b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:15.720360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:15.722028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:15.722920 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:15.723953 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:15.737683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:15.739346 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:15.761415 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1398/2783] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:16.092843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:16.094285 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:16.114629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126e9b3-184e-40f2-9c67-a1702231b64b None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:16.124733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:16.125785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:16.126205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:16.126629 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] PATH_INFO: `/v3/projects/81fec515e17d412398ade6c10c71fa72` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:16.128088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81fec515e17d412398ade6c10c71fa72)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:16.130165 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:16.181769 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:16.186345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81fec515e17d412398ade6c10c71fa72', 'request_id': 'req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.197558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81fec515e17d412398ade6c10c71fa72', 'request_id': 'req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.203910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81fec515e17d412398ade6c10c71fa72', 'request_id': 'req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.218301 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] The token cache is being invalidate because project 81fec515e17d412398ade6c10c71fa72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:16.218683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.219437 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.221087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1386/2784] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:16 2022] DELETE /identity/v3/projects/81fec515e17d412398ade6c10c71fa72 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:16.225419 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:16.236299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46235b23-5297-4cdc-a884-718e990916f3 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:16.307620 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:16.308364 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:16.308743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:16.309083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] PATH_INFO: `/v3/users/834e7db0d8a4438e81a35b148cad4954` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:16.310311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=834e7db0d8a4438e81a35b148cad4954)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:16.312673 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:16.385802 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834e7db0d8a4438e81a35b148cad4954'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.406235 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834e7db0d8a4438e81a35b148cad4954'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.413694 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834e7db0d8a4438e81a35b148cad4954'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.414781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834e7db0d8a4438e81a35b148cad4954'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.415837 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-639d91bf-4d28-4968-8b7f-ce76e6509bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834e7db0d8a4438e81a35b148cad4954'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:16.432379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1399/2785] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:16 2022] DELETE /identity/v3/users/834e7db0d8a4438e81a35b148cad4954 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:16.945848 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:16.947468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:16.976882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdac1-0c06-45eb-bc94-fe15e3d54d05 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.085514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.086079 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.086358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.086627 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] PATH_INFO: `/v3/projects/1e32d292f23d48e48185e99897aa1f25` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.096956 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e32d292f23d48e48185e99897aa1f25)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.099914 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.194890 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e32d292f23d48e48185e99897aa1f25', 'request_id': 'req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.210215 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e32d292f23d48e48185e99897aa1f25', 'request_id': 'req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.220382 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e32d292f23d48e48185e99897aa1f25', 'request_id': 'req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.243166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] The token cache is being invalidate because project 1e32d292f23d48e48185e99897aa1f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:17.243786 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.244690 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-b4b29b4a-4332-480b-8b9c-aab4a418f9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.257613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1387/2786] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:16 2022] DELETE /identity/v3/projects/1e32d292f23d48e48185e99897aa1f25 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:17.260679 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.264108 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.264394 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.289740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:17.291048 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.381160 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1400/2787] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:17.390080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.390523 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.390852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.398161 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.399873 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.415498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbe2e6-04a4-463a-bcb5-f0fb1805e13b None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.420256 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.420436 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:17.420981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.421331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.421647 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.421799 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] PATH_INFO: `/v3/users/687629f84f1f408d8e80a5a1b28eb787` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.422941 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687629f84f1f408d8e80a5a1b28eb787)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.425012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.426754 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:17.430715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1388/2788] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:17.434999 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.445840 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854336da-f7fb-4d13-8bad-b1cd83a06d11 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.450511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.451159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.451456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.451702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.452348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:17.452653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.454398 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.467516 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1389/2789] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 03:57:17.472633 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.483060 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa21e60-c181-427d-8f75-3adfda39643b admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.488083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.488679 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.488955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.489220 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.489472 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687629f84f1f408d8e80a5a1b28eb787'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.489860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.491779 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.502373 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687629f84f1f408d8e80a5a1b28eb787'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.509829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687629f84f1f408d8e80a5a1b28eb787'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.510613 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687629f84f1f408d8e80a5a1b28eb787'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.511507 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687629f84f1f408d8e80a5a1b28eb787'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.526654 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1390/2790] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:17 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:17.527680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.527932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1401/2791] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:17 2022] DELETE /identity/v3/users/687629f84f1f408d8e80a5a1b28eb787 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:17.532591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.556918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:17.610267 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b74d94-ecd8-4a5b-8334-e5168650940c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.676765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.677653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.677939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.678219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.678895 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.680849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.717797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1391/2792] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:17 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:17.720637 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.726416 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.728265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.735644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b9d3-5366-4861-9680-700bfd6ecbf5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.742286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.743034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.743420 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.743553 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842d56-1fa6-4ee5-ab16-8f5ef589138d admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.743834 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.744609 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.746321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.748881 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.749514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.749787 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.750077 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] PATH_INFO: `/v3/projects/5ced06c15af249bf92f546d0a0bfba23` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.751457 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ced06c15af249bf92f546d0a0bfba23)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.753402 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.762418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1402/2793] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:17.766724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.777687 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c358815-8c47-4227-b270-94b281a858f2 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.782931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.783595 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.783905 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.784214 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.784913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.786740 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.797923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1403/2794] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:17.802955 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.813615 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcabeff-b162-4dd2-b5b5-4e3112c816ff admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.815706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ced06c15af249bf92f546d0a0bfba23', 'request_id': 'req-83909454-f402-4029-8162-3b0389d50c00'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.825163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ced06c15af249bf92f546d0a0bfba23', 'request_id': 'req-83909454-f402-4029-8162-3b0389d50c00'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.831222 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ced06c15af249bf92f546d0a0bfba23', 'request_id': 'req-83909454-f402-4029-8162-3b0389d50c00'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.845031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] The token cache is being invalidate because project 5ced06c15af249bf92f546d0a0bfba23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:17.845340 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.846059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:17.847642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1392/2795] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:17 2022] DELETE /identity/v3/projects/5ced06c15af249bf92f546d0a0bfba23 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:17.851515 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:17.852899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.878610 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.879293 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.879577 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.879838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] PATH_INFO: `/v3/projects/7fd13b3c769c466b912c4f4f64e08a07/users/206956ffc02845f7b960a1cf85b9cd54/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.894524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd13b3c769c466b912c4f4f64e08a07, user_id=206956ffc02845f7b960a1cf85b9cd54, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.897304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9f352-d9f0-4d27-a21b-816e0da04d8f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:17.901861 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:17.914182 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1404/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:17 2022] PUT /identity/v3/projects/7fd13b3c769c466b912c4f4f64e08a07/users/206956ffc02845f7b960a1cf85b9cd54/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:17.924440 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.924758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.925029 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.929081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909454-f402-4029-8162-3b0389d50c00 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:17.953891 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] MFA Rules not processed for user `206956ffc02845f7b960a1cf85b9cd54`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:17.955144 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:17.987665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:17.988311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:17.988599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:17.988873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] PATH_INFO: `/v3/users/4b3d38cf375547558829cbe46b7a62e5` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:17.990039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3d38cf375547558829cbe46b7a62e5)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:17.991843 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:17.992042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.031314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1405/2797] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:18.036098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.041913 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3d38cf375547558829cbe46b7a62e5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.042660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.042817 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3d38cf375547558829cbe46b7a62e5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.054869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3d38cf375547558829cbe46b7a62e5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.057022 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52c694-dbae-4e00-b724-b0ddfb3361c8 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.061407 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.062019 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.062295 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.062521 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3d38cf375547558829cbe46b7a62e5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.062643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.063247 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.063461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3d38cf375547558829cbe46b7a62e5'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.065289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.077646 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1393/2798] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:17 2022] DELETE /identity/v3/users/4b3d38cf375547558829cbe46b7a62e5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:18.094552 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1406/2799] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:18 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:18.099179 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.100613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.174613 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564ec51-437c-4a1e-82e3-9f8473795e72 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.185245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:18.255598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.256210 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.256696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.257107 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.262083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.264569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.282284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.282284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.300060 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03686-68af-48ee-931b-a0c0f4471754 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.303831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1394/2800] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:18 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:18.305499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.306222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.306557 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.306933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] PATH_INFO: `/v3/projects/d62871dac47946038b561e2467c53461` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.307298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.308167 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62871dac47946038b561e2467c53461)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.310069 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.317934 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364978e0-a6a7-47a9-a8b8-1f6955e9ac66 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.322761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.323480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.323764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.324038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.324683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.326274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.342526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1395/2801] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:18.346202 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.356737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cf18c-9f59-4a7f-81e8-935e1135bbce admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.361643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62871dac47946038b561e2467c53461', 'request_id': 'req-9b898e24-51f6-41f2-a640-91799cd9e329'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.370209 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62871dac47946038b561e2467c53461', 'request_id': 'req-9b898e24-51f6-41f2-a640-91799cd9e329'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.376382 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62871dac47946038b561e2467c53461', 'request_id': 'req-9b898e24-51f6-41f2-a640-91799cd9e329'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.390332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] The token cache is being invalidate because project d62871dac47946038b561e2467c53461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:18.390635 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.392101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:18.406855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1407/2802] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:18 2022] DELETE /identity/v3/projects/d62871dac47946038b561e2467c53461 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:18.432843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.433580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.433940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.434268 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.435031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.436838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af228d3-5d70-4bb2-813f-9f630e68bf24 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.449310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1396/2803] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:18.457768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.459683 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.480398 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898e24-51f6-41f2-a640-91799cd9e329 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.495645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.496511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.496887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.497308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] PATH_INFO: `/v3/projects/4abd627584a64df49cd64acdf6e5d08d/users/f3a4cf4744d34062b9374c4cf24442ff/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.519658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abd627584a64df49cd64acdf6e5d08d, user_id=f3a4cf4744d34062b9374c4cf24442ff, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.523237 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.538621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1408/2804] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:18 2022] PUT /identity/v3/projects/4abd627584a64df49cd64acdf6e5d08d/users/f3a4cf4744d34062b9374c4cf24442ff/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:18.611264 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.611524 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.611849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.644749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] MFA Rules not processed for user `f3a4cf4744d34062b9374c4cf24442ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:18.645993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.716275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1397/2805] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:18.720909 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.739152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.740574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.770985 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:18.807458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91059c1f-e6c7-4d3e-92e4-b9bbad25edb4 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.812242 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.812861 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.813154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.813418 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.814066 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.816100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.827864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:18.835078 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3a06-9a9b-4259-b365-9792cced1061 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.841345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.841918 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.842196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.842468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.844204 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:18.848416 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1409/2806] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:18.851666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.856425 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.858023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.868716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1398/2807] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:18.915927 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652e44f-1581-41e7-8a52-5b542b1a6559 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.921874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.922495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.922773 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.923076 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.923715 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.925688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.954203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1410/2808] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:18 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:18.955505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.966180 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba00185-716d-4d12-93cd-c99e060447d0 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:18.971054 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:18.971595 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:18.971878 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:18.972152 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:18.972801 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:18.974405 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:18.984725 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1399/2809] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:18.989002 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:18.999354 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26bc-a188-449c-8c51-0d0559a582c7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:19.004327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:19.005016 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:19.005273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:19.005523 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] PATH_INFO: `/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c/users/226e9fa6c80d4723b886770be3bc2a26/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:19.025254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eff2c4dd3a64d4bad90e1adb5dfc93c, user_id=226e9fa6c80d4723b886770be3bc2a26, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:19.028044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:19.044126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1411/2810] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:18 2022] PUT /identity/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c/users/226e9fa6c80d4723b886770be3bc2a26/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:19.048032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:19.058622 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399940e5-5e6d-4604-bed4-777851ef2f93 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:19.121367 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:19.121948 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:19.122227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:19.122509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:19.123201 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:19.124829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab40dbe-239d-4c1e-991a-c061afed7c0c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:19.134823 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1400/2811] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:19.140692 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:19.152028 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb77-9baf-4d35-85d9-461058e18148 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:19.157331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:19.158071 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:19.158387 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:19.158656 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] PATH_INFO: `/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c/users/226e9fa6c80d4723b886770be3bc2a26/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:19.163018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eff2c4dd3a64d4bad90e1adb5dfc93c, user_id=226e9fa6c80d4723b886770be3bc2a26, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:19.166020 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:19.178564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1412/2812] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:19 2022] PUT /identity/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c/users/226e9fa6c80d4723b886770be3bc2a26/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:19.189928 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:19.190330 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:19.190744 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:19.231134 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] MFA Rules not processed for user `226e9fa6c80d4723b886770be3bc2a26`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:19.232265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:19.303792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1401/2813] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:19.317095 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:19.319197 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:19.382395 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af4e9c-b557-4cd9-8ffd-26f7c225f3b1 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:19.386578 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:19.387241 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:19.387519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:19.387789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:19.389820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:19.460438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:19.461876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:19.465359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:19.472964 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1413/2814] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:21.018819 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:21.020327 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:21.032680 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c454e21-d91b-4535-9c5e-bab72188ef81 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:21.036427 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:21.037023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:21.037293 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:21.037558 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:21.050486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:21.100342 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:21.101749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:21.112106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1402/2815] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:21.718422 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:21.728004 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ff96a-103a-4521-bbde-4c529d1febce service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:21.732876 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:21.733505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:21.733785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:21.734055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:21.744025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:21.745445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325a50a-da53-4a7f-99e5-e500ec263312 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:21.755222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1414/2816] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:25.703955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:25.774737 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f5c0-6ac8-4ea7-aa9e-c7d5835e9004 service cinder] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:25.783297 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:25.783955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:25.784236 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:25.784507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] PATH_INFO: `/v3/users/206956ffc02845f7b960a1cf85b9cd54` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:25.785768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206956ffc02845f7b960a1cf85b9cd54)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:25.787803 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:25.842418 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206956ffc02845f7b960a1cf85b9cd54'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:25.843044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206956ffc02845f7b960a1cf85b9cd54'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:25.855424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206956ffc02845f7b960a1cf85b9cd54'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:25.863621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206956ffc02845f7b960a1cf85b9cd54'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:25.864248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206956ffc02845f7b960a1cf85b9cd54'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:25.881557 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1403/2817] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:25 2022] DELETE /identity/v3/users/206956ffc02845f7b960a1cf85b9cd54 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:25.887979 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:25.888493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:25.888608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:25.909433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:25.910654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:25.972154 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1415/2818] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:25.982686 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:25.984300 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:26.024384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:26.065053 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75e639-ba75-436a-a18d-002ecff59978 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:26.069372 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:26.070015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:26.070296 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:26.070574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:26.080762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:26.082217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:26.092090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1404/2819] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:26.369381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:26.371079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:26.383250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582a551-b8c0-418f-a8b6-09418ba0acdc None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:26.389272 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:26.390041 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:26.390339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:26.390705 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] PATH_INFO: `/v3/projects/7fd13b3c769c466b912c4f4f64e08a07` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:26.391753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd13b3c769c466b912c4f4f64e08a07)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:26.394461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:26.445457 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd13b3c769c466b912c4f4f64e08a07', 'request_id': 'req-d513fc02-848b-4967-ab38-9fb480590b23'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.452505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd13b3c769c466b912c4f4f64e08a07', 'request_id': 'req-d513fc02-848b-4967-ab38-9fb480590b23'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.458009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd13b3c769c466b912c4f4f64e08a07', 'request_id': 'req-d513fc02-848b-4967-ab38-9fb480590b23'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.469180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] The token cache is being invalidate because project 7fd13b3c769c466b912c4f4f64e08a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:26.469467 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.470165 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.472074 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1416/2820] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:26 2022] DELETE /identity/v3/projects/7fd13b3c769c466b912c4f4f64e08a07 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:26.476373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:26.486873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eefa2-e1c0-4e13-82f1-ed2a56d5be63 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:26.550665 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:26.551392 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:26.551673 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:26.551942 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] PATH_INFO: `/v3/users/f3a4cf4744d34062b9374c4cf24442ff` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:26.553070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a4cf4744d34062b9374c4cf24442ff)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:26.555033 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:26.605640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a4cf4744d34062b9374c4cf24442ff'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.606387 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a4cf4744d34062b9374c4cf24442ff'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.619730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a4cf4744d34062b9374c4cf24442ff'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.626348 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a4cf4744d34062b9374c4cf24442ff'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.627025 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a4cf4744d34062b9374c4cf24442ff'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:26.641497 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1405/2821] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:26 2022] DELETE /identity/v3/users/f3a4cf4744d34062b9374c4cf24442ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:26.927117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:26.929057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:26.945736 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513fc02-848b-4967-ab38-9fb480590b23 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:27.006348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:27.007034 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.007315 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.007585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] PATH_INFO: `/v3/projects/4abd627584a64df49cd64acdf6e5d08d` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.008724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abd627584a64df49cd64acdf6e5d08d)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:27.010642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:27.021047 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:27.059298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abd627584a64df49cd64acdf6e5d08d', 'request_id': 'req-2b4fae1f-416d-430c-9d41-43df0819a42a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.068223 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abd627584a64df49cd64acdf6e5d08d', 'request_id': 'req-2b4fae1f-416d-430c-9d41-43df0819a42a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.074222 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abd627584a64df49cd64acdf6e5d08d', 'request_id': 'req-2b4fae1f-416d-430c-9d41-43df0819a42a'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.086277 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] The token cache is being invalidate because project 4abd627584a64df49cd64acdf6e5d08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:27.086564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.087310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.088871 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1417/2822] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:26 2022] DELETE /identity/v3/projects/4abd627584a64df49cd64acdf6e5d08d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:27.093131 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:27.094628 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:27.108321 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f601083-e197-411d-969c-b61448781a5f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:27.142198 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:27.167582 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:27.168226 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.168501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.168755 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] PATH_INFO: `/v3/users/226e9fa6c80d4723b886770be3bc2a26` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.169920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226e9fa6c80d4723b886770be3bc2a26)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:27.171836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:27.238023 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226e9fa6c80d4723b886770be3bc2a26'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.238685 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226e9fa6c80d4723b886770be3bc2a26'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.252038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226e9fa6c80d4723b886770be3bc2a26'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.261849 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226e9fa6c80d4723b886770be3bc2a26'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.262667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-35a17da5-c68b-41c5-989d-4dcdb8769fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226e9fa6c80d4723b886770be3bc2a26'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.276581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1406/2823] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:27 2022] DELETE /identity/v3/users/226e9fa6c80d4723b886770be3bc2a26 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:27.567864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:27.569801 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:27.587036 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fae1f-416d-430c-9d41-43df0819a42a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:27.644216 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:27.644854 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.645137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.645405 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] PATH_INFO: `/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.646572 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eff2c4dd3a64d4bad90e1adb5dfc93c)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:27.648504 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:27.697238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eff2c4dd3a64d4bad90e1adb5dfc93c', 'request_id': 'req-535c5adc-3048-4b50-b72c-d3772eccad26'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.704727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eff2c4dd3a64d4bad90e1adb5dfc93c', 'request_id': 'req-535c5adc-3048-4b50-b72c-d3772eccad26'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.710447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eff2c4dd3a64d4bad90e1adb5dfc93c', 'request_id': 'req-535c5adc-3048-4b50-b72c-d3772eccad26'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.721654 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] The token cache is being invalidate because project 2eff2c4dd3a64d4bad90e1adb5dfc93c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:27.721942 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.722644 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-535c5adc-3048-4b50-b72c-d3772eccad26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:27.724204 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1418/2824] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:27 2022] DELETE /identity/v3/projects/2eff2c4dd3a64d4bad90e1adb5dfc93c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:27.729378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.729667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.729940 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.751764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:27.752975 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:27.814072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1407/2825] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:27.825284 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.825746 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.826044 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.852220 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:27.853421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:27.862734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1419/2826] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:27.869644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:27.882506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f502d1-3a72-4d1a-83f4-ef8fecb38383 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:27.888083 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:27.888693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.888973 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.889246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.889894 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:27.890174 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:27.891831 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:27.903700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1408/2827] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:27.908340 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:27.917967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c80283-a752-4b3c-97c8-f24fee809a50 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:27.923013 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:27.923636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:27.923907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:27.924191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:27.924832 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:27.926764 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:27.952145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:27.959952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1420/2828] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:27 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:27.964542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:27.977346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410734-ab64-40c1-8c72-68b19c980e52 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:28.024500 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:28.043688 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:28.044302 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.044580 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.044850 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.045522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:28.047506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:28.072217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1409/2829] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:27 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:28.078339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:28.089231 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537ee1e-9a6a-437e-86b6-acb56dff02be admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:28.094481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:28.095118 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.095433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.095708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.096360 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:28.097965 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:28.108684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1421/2830] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:28.113016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:28.123693 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99a2a-8a0f-4065-827f-143699765eeb admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:28.129371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:28.129965 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.130239 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.130507 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.131194 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:28.132804 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1b95-9f97-4103-954a-ec8957e9d675 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:28.145090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1410/2831] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:28.149348 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:28.159714 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f6db-d4bb-4383-a152-78296c1fe05d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:28.164729 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:28.165379 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.165655 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.165949 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] PATH_INFO: `/v3/projects/83871ae750f64fab9bc96df6f8c01647/users/880fd79e0b9242dd8c9e20f65af6eb9d/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.180988 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83871ae750f64fab9bc96df6f8c01647, user_id=880fd79e0b9242dd8c9e20f65af6eb9d, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:28.183710 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:28.201981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1422/2832] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:28 2022] PUT /identity/v3/projects/83871ae750f64fab9bc96df6f8c01647/users/880fd79e0b9242dd8c9e20f65af6eb9d/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:28.214129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.214373 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.214645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.371480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] MFA Rules not processed for user `880fd79e0b9242dd8c9e20f65af6eb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:28.372696 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:28.436903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1411/2833] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:28.451967 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:28.454021 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:28.522479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60849b22-4953-429b-9ded-bc725a57da44 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:28.526719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:28.527362 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:28.527631 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:28.527901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:28.537970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:28.539447 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c87a0-eca6-414a-bcd6-4b52a4878e32 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:28.548563 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1423/2834] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:34.166123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:34.166123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:34.226167 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87028342-1c4a-4d61-983e-b1b02dd217ce None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:34.231797 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:34.232438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:34.232713 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:34.232982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] PATH_INFO: `/v3/users/afb9eb7271564fd48a88cb94fd0b7991` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:34.234156 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb9eb7271564fd48a88cb94fd0b7991)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:34.236081 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:34.287344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb9eb7271564fd48a88cb94fd0b7991'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.288021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb9eb7271564fd48a88cb94fd0b7991'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.293205 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:34.301067 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb9eb7271564fd48a88cb94fd0b7991'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.311019 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb9eb7271564fd48a88cb94fd0b7991'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.311982 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb9eb7271564fd48a88cb94fd0b7991'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.327702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1412/2835] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:34 2022] DELETE /identity/v3/users/afb9eb7271564fd48a88cb94fd0b7991 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:34.332495 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:34.333006 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:34.333119 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:34.352392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:34.353583 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:34.416063 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1424/2836] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:34.426026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:34.427681 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:34.486456 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fde-d7d8-458e-acfb-ed7763cc1123 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:34.494038 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:34.494925 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:34.495910 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:34.496345 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:34.508116 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:34.509593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf9131-4093-46ab-ba11-d0a03017dffe service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:34.519094 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1413/2837] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:34.800334 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:34.802077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:34.815083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aa644-23a9-429b-96a1-9692d20cf2e1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:34.820432 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:34.821839 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:34.822532 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:34.823031 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] PATH_INFO: `/v3/projects/e06555bf54b2487cb91ebf22a987d416` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:34.824326 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06555bf54b2487cb91ebf22a987d416)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:34.826719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:34.879939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06555bf54b2487cb91ebf22a987d416', 'request_id': 'req-d0d72c09-89f1-45ac-afe6-ca99aa677507'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.889473 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06555bf54b2487cb91ebf22a987d416', 'request_id': 'req-d0d72c09-89f1-45ac-afe6-ca99aa677507'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.894669 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:34.895947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06555bf54b2487cb91ebf22a987d416', 'request_id': 'req-d0d72c09-89f1-45ac-afe6-ca99aa677507'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.908316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] The token cache is being invalidate because project e06555bf54b2487cb91ebf22a987d416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:34.908599 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.909238 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d0d72c09-89f1-45ac-afe6-ca99aa677507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:34.910727 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1425/2838] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:34 2022] DELETE /identity/v3/projects/e06555bf54b2487cb91ebf22a987d416 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:34.915724 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:34.916008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:34.916274 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:34.936263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:34.937493 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.000248 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1414/2839] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:35.064899 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.065368 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.065819 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.087875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:35.089098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.098824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1426/2840] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:35.103271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.114535 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e60427-e203-4aa2-abf7-63a7e4887f85 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.120538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.121136 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.121410 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.121671 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.122310 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:35.122610 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.124263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.136187 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1415/2841] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:35.140587 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.150838 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c757-0c3d-42bb-9c58-865bbf31f6c2 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.156313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.156904 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.157205 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.157475 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.158117 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.160045 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.190384 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1427/2842] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:35 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:35.195024 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.205143 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ccd4e-e7b6-4ee1-ac13-aa071c8c0a6f admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.264355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.264981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.265265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.265538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.266186 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.268168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.277343 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:35.292891 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1416/2843] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:35 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:35.297261 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.308015 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e8dfb-2c02-4482-a3e8-965203b348b0 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.313246 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.313848 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.314126 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.314396 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.315061 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.316652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.326913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1428/2844] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:35.330938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.343903 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a54b4-db9c-4f06-9d66-97bf3d0b59ce admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.351479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.352084 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.352356 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.352623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.353256 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.355218 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c2ec4-bebf-4eeb-b862-22397febee3b admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.366129 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1417/2845] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:35.370742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.386249 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51842d1-ab6b-4fa6-accb-9198f7a3140d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.394300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.395310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.395666 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.395938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] PATH_INFO: `/v3/projects/fb3bafd036fc40bfa31f68d948bae504/users/c869a024c8504f2d8dd6121432b5e81f/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.411101 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3bafd036fc40bfa31f68d948bae504, user_id=c869a024c8504f2d8dd6121432b5e81f, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.413743 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.428929 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1429/2846] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:35 2022] PUT /identity/v3/projects/fb3bafd036fc40bfa31f68d948bae504/users/c869a024c8504f2d8dd6121432b5e81f/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:35.439829 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.440118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.440467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.476163 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] MFA Rules not processed for user `c869a024c8504f2d8dd6121432b5e81f`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:35.477371 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.542072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1418/2847] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:35.552940 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.555186 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.634708 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.636166 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.636852 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e893-ab5d-4cb2-84b9-dda1a4708e5c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.640880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.641506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.641779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.642049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.650555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.652068 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af390cf1-0a89-4f8b-a22a-a3e2c753b049 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.661322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1430/2848] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:35.703584 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876630cd-8571-4ea6-951f-d56a910fe23c None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.708404 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.709045 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.709338 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.709608 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] PATH_INFO: `/v3/users/880fd79e0b9242dd8c9e20f65af6eb9d` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.710712 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880fd79e0b9242dd8c9e20f65af6eb9d)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.712661 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.761564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880fd79e0b9242dd8c9e20f65af6eb9d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:35.762341 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880fd79e0b9242dd8c9e20f65af6eb9d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:35.774150 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880fd79e0b9242dd8c9e20f65af6eb9d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:35.779549 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:35.782375 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880fd79e0b9242dd8c9e20f65af6eb9d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:35.783223 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880fd79e0b9242dd8c9e20f65af6eb9d'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:35.797573 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1419/2849] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:35 2022] DELETE /identity/v3/users/880fd79e0b9242dd8c9e20f65af6eb9d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:35.802319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.802658 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.802970 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.827787 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:35.829007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.894408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1431/2850] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:35.904231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:35.905943 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:35.973227 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050685b-5806-4ab4-8335-b17f0ddc2055 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:35.976960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:35.977564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:35.977839 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:35.978105 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:35.987706 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:35.989266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd82d-7e29-45eb-a803-f82c6db6f6c3 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:35.999993 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1420/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:36.267939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.269476 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:36.284789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8b851-9f22-4904-9994-8cb310b6ec7f None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.290273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.291064 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.291433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.291923 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] PATH_INFO: `/v3/projects/83871ae750f64fab9bc96df6f8c01647` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.293780 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=83871ae750f64fab9bc96df6f8c01647)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.295794 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.303624 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:36.347245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83871ae750f64fab9bc96df6f8c01647', 'request_id': 'req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:36.355668 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83871ae750f64fab9bc96df6f8c01647', 'request_id': 'req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:36.361146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83871ae750f64fab9bc96df6f8c01647', 'request_id': 'req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:36.372321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] The token cache is being invalidate because project 83871ae750f64fab9bc96df6f8c01647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:36.372938 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:36.373269 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-d2c1d22d-896a-4d71-80a8-7cea5b04b4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:36.374774 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1432/2852] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:36 2022] DELETE /identity/v3/projects/83871ae750f64fab9bc96df6f8c01647 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:36.379798 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.380265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.380374 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.400031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:36.401232 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:36.460984 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1421/2853] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:36.468742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.469056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.469327 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.491260 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:36.492461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:36.500977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1433/2854] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:36.505765 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.520843 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f5b9f-347e-428f-a458-cae068ce79c6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.529631 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.530445 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.530815 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.531138 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.532097 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:36.532438 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.534717 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.547013 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1422/2855] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:36.551426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.560786 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e08590-a475-43ae-b0ae-a0ba765becf0 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.565986 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.566586 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.566885 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.567153 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.567816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.569697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.598141 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1434/2856] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:36 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:36.602276 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.612576 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd454-b2f8-469c-9853-0241c419c246 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.670332 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.670976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.671252 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.671525 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.672150 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.674072 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.696229 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1423/2857] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:36 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:36.700569 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.711030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdede6-b13b-45a2-91ac-d0d8e88e7343 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.715936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.716529 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.716800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.717065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.717700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.719308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.729352 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1435/2858] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:36.733988 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.746752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1160960-7874-464b-ae47-e60072d41acc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.756981 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.758005 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.758363 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.758771 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.759822 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.762397 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b663122-e7d0-4cf6-86bf-6b1ab86f56de admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.765818 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:36.779253 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1424/2859] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:36.783665 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.794090 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4a10-70d7-4ba0-9a94-e1f7ab34f806 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:36.799767 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:36.800421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.800726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.801025 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] PATH_INFO: `/v3/projects/766e95a843fd4c658750c125427135cf/users/0a5317994a7443fe8baeff831cd9ee81/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.816721 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=766e95a843fd4c658750c125427135cf, user_id=0a5317994a7443fe8baeff831cd9ee81, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:36.819466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:36.834503 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1436/2860] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:36 2022] PUT /identity/v3/projects/766e95a843fd4c658750c125427135cf/users/0a5317994a7443fe8baeff831cd9ee81/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:36.839602 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:36.839885 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:36.840162 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:36.860157 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:36.861322 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:36.923916 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1425/2861] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:36.936408 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:36.938511 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:37.000433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abed5a-4c56-4b57-b23f-6bf6ed5fb1f5 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:37.004438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:37.005098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:37.005431 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:37.005726 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:37.014698 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:37.016176 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc70a6c-cf27-4d65-a427-f6c3556d527a service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:37.025908 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1437/2862] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:38.755317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:38.756922 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:38.768882 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740fb2f-ba86-43e6-bd82-a729868dbe4f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:38.774016 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:38.774677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:38.775103 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:38.775422 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] PATH_INFO: `/v3/users/c869a024c8504f2d8dd6121432b5e81f` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:38.776623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c869a024c8504f2d8dd6121432b5e81f)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:38.778612 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:38.828376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c869a024c8504f2d8dd6121432b5e81f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:38.829070 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c869a024c8504f2d8dd6121432b5e81f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:38.843181 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c869a024c8504f2d8dd6121432b5e81f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:38.851731 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c869a024c8504f2d8dd6121432b5e81f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:38.852478 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c869a024c8504f2d8dd6121432b5e81f'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:38.868433 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1426/2863] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:38 2022] DELETE /identity/v3/users/c869a024c8504f2d8dd6121432b5e81f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:38.872954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:38.873468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:38.873468 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:38.895700 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:38.896936 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:38.933003 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:38.977554 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1438/2864] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:38.988793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:38.990621 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:39.067446 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8bf5-d83b-412d-b33e-a09a4aa6d90c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:39.071687 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:39.072396 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.072733 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.072992 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.086231 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:39.088006 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c465a-bb72-42fe-888a-2d0d520d26aa service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:39.100630 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1427/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:39.434973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:39.437724 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:39.459509 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e026-e9ec-47e7-8447-e588c5ac1128 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:39.465017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:39.465742 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.466358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.466358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] PATH_INFO: `/v3/projects/fb3bafd036fc40bfa31f68d948bae504` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.467518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3bafd036fc40bfa31f68d948bae504)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:39.469702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:39.578878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3bafd036fc40bfa31f68d948bae504', 'request_id': 'req-92de427f-6e1c-4274-947c-3cd20c509cbe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:39.589697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3bafd036fc40bfa31f68d948bae504', 'request_id': 'req-92de427f-6e1c-4274-947c-3cd20c509cbe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:39.600080 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3bafd036fc40bfa31f68d948bae504', 'request_id': 'req-92de427f-6e1c-4274-947c-3cd20c509cbe'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:39.620689 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] The token cache is being invalidate because project fb3bafd036fc40bfa31f68d948bae504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:39.624367 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:39.625713 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-92de427f-6e1c-4274-947c-3cd20c509cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:39.631800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1439/2866] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:39 2022] DELETE /identity/v3/projects/fb3bafd036fc40bfa31f68d948bae504 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:39.636836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.637358 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.638259 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.664477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:39.665800 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:39.681260 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:39.758964 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1428/2867] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:39.765901 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.766192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.766455 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.791168 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:39.792811 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:39.801636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1440/2868] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:39.806126 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:39.816522 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0a92d-cc0d-44ce-b34f-1d2099e1049f None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:39.821768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:39.822604 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.822924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.823531 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.823855 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93069) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:39.824159 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:39.825824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:39.841003 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1429/2869] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:39.845745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:39.856479 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2aaf-38b9-40e7-bd6f-56ee68bedae1 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:39.868393 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:39.869401 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:39.869841 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:39.870287 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:39.871163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:39.873732 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:39.917308 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:39.917530 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1441/2870] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:39 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:39.929773 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f577c35-df8a-4443-8a02-0a88daf3dec5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:40.006394 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:40.006979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.007286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.007561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.008235 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:40.010331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:40.035176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1430/2871] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:39 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:40.039298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:40.049982 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db1ac-e4ea-4ff4-b561-244125c029ea admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:40.055660 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:40.056545 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.056989 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.057435 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.058331 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:40.060172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:40.071865 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1442/2872] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:40.076477 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:40.090660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3d72-ed43-461e-93fb-7cc02a648a1c admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:40.095415 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:40.096039 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.096319 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.096589 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.097246 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:40.099147 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8a18-1200-41f5-ba19-33e2f3b7fbb3 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:40.115050 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:40.115660 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1431/2873] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:40.125851 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338bdf3-124a-4876-bd11-cade08819086 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:40.130702 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:40.131480 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.131826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.132173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] PATH_INFO: `/v3/projects/e18dba64599d4bcdab480e7239291865/users/984915b5c86441c09d49e39e44a005fd/roles/87b7a008126642369351bd284f3e8354` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.148878 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18dba64599d4bcdab480e7239291865, user_id=984915b5c86441c09d49e39e44a005fd, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:40.151718 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:40.169324 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1443/2874] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:40 2022] PUT /identity/v3/projects/e18dba64599d4bcdab480e7239291865/users/984915b5c86441c09d49e39e44a005fd/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 03:57:40.179980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddae0-7640-48a9-9465-b23ea63dd91e None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.180510 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddae0-7640-48a9-9465-b23ea63dd91e None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.180979 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddae0-7640-48a9-9465-b23ea63dd91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.213909 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-c54ddae0-7640-48a9-9465-b23ea63dd91e None None] MFA Rules not processed for user `984915b5c86441c09d49e39e44a005fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:40.215224 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c54ddae0-7640-48a9-9465-b23ea63dd91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:40.288032 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1432/2875] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:40.301409 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:40.303527 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:40.371318 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c6d05-62bd-49b6-8951-ce2774b4d3bf admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:40.375862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09249c2527c148cd99ebf899022f76fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:40.376298 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:40.376645 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:40.377014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:40.385778 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:40.389547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:40.391322 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:40.406985 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1444/2876] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:41.541377 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:41.541929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:41.541929 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:41.564700 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:41.576569 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] MFA Rules not processed for user `0a5317994a7443fe8baeff831cd9ee81`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:41.577935 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:41.644715 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1433/2877] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:41.649313 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:41.650762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:41.718963 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176deb97-81cb-4310-8c31-0f8863d57367 service cinder] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:41.723559 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:41.724164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:41.724438 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:41.724703 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] PATH_INFO: `/v3/projects` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:41.725323 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:41.727274 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:41.754757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1445/2878] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:41 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:41.758924 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:41.770852 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48007e2-8a55-4dc7-b34b-230c0196a4c9 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:41.830741 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:41.831329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:41.831635 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:41.831906 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] PATH_INFO: `/v3/users` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:41.832547 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:41.834603 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:41.858728 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1434/2879] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:41 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:41.863137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:41.873820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8c9b-67dc-4a09-9166-97bbcfa6e26e admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:41.879033 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:41.880203 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:41.880555 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:41.881187 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:41.881951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:41.883987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:41.904053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1446/2880] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 03:57:41.908530 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:41.918792 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339018-629d-409a-870f-f3fc9c117bca admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:41.924028 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:41.924794 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:41.925099 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:41.925376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] PATH_INFO: `/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74/users/ee1094de309444bd9010b33ced748435/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:41.941563 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a66ef2ac1840d2b030b0f059bebf74, user_id=ee1094de309444bd9010b33ced748435, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:41.944272 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:41.958870 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1435/2881] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:41 2022] PUT /identity/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74/users/ee1094de309444bd9010b33ced748435/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:41.964962 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:41.973632 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe98e-76b0-4c9b-9e2c-26ee5369ed40 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:42.045862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:42.046548 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:42.047077 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:42.047199 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:42.047766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:42.049388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:42.060386 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1447/2882] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:42.064701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:42.076643 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0ba6-9954-4b9c-8462-90e9879c0375 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:42.082298 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:42.083121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:42.083472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:42.083980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] PATH_INFO: `/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74/users/ee1094de309444bd9010b33ced748435/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:42.087089 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a66ef2ac1840d2b030b0f059bebf74, user_id=ee1094de309444bd9010b33ced748435, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:42.092228 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232faa-d7ee-4f75-84db-e0ce09036f98 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:42.104594 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1436/2883] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:42 2022] PUT /identity/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74/users/ee1094de309444bd9010b33ced748435/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:43.108725 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:43.120124 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80e993-1929-40b4-afe6-f9c32f1497db admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:43.210825 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:43.211717 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:43.212000 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:43.212270 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] PATH_INFO: `/v3/users/984915b5c86441c09d49e39e44a005fd` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:43.213546 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=984915b5c86441c09d49e39e44a005fd)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:43.216734 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:43.226173 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:43.287319 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984915b5c86441c09d49e39e44a005fd'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:43.301950 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984915b5c86441c09d49e39e44a005fd'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:43.310932 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984915b5c86441c09d49e39e44a005fd'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:43.311877 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984915b5c86441c09d49e39e44a005fd'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:43.312641 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984915b5c86441c09d49e39e44a005fd'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:43.330453 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1448/2884] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 03:57:43 2022] DELETE /identity/v3/users/984915b5c86441c09d49e39e44a005fd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:43.335238 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:43.335751 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:43.335869 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:43.362710 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:43.364384 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:43.417504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:43.435419 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1437/2885] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:43.444057 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:43.446512 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:43.519652 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb17b3-32f8-440e-ad12-a27245afdbf8 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:43.524506 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:43.525499 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:43.525779 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:43.526137 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:43.537245 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:43.538762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce7cd-705e-4e4c-aef6-f2453fb6828f service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:43.550497 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1449/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:44.054876 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.056626 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:44.070245 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248609-ed77-46b2-bd4c-d4e29a8cac30 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.076088 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.076767 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.077043 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.077314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] PATH_INFO: `/v3/projects/e18dba64599d4bcdab480e7239291865` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.078699 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18dba64599d4bcdab480e7239291865)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.080768 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.155034 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18dba64599d4bcdab480e7239291865', 'request_id': 'req-bc5173b7-e7bf-47d9-8186-954a40e0815e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:44.167106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18dba64599d4bcdab480e7239291865', 'request_id': 'req-bc5173b7-e7bf-47d9-8186-954a40e0815e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:44.175376 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18dba64599d4bcdab480e7239291865', 'request_id': 'req-bc5173b7-e7bf-47d9-8186-954a40e0815e'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:44.196574 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] The token cache is being invalidate because project e18dba64599d4bcdab480e7239291865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 03:57:44.196836 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:44.197659 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc5173b7-e7bf-47d9-8186-954a40e0815e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 03:57:44.200562 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1438/2887] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 03:57:44 2022] DELETE /identity/v3/projects/e18dba64599d4bcdab480e7239291865 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:44.208937 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.209493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.209493 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.240859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:44.244009 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:44.321670 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1450/2888] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:44.330640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.330640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.330640 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.366146 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:44.367431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:44.392899 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1439/2889] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:44.397286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.410797 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195ea6b-7d60-4da8-8dea-c7d4f6f59621 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.428684 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.431619 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.431792 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.432109 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.432880 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93068) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 03:57:44.433282 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.459375 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.520224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1451/2890] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Sep 3 03:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 03:57:44.521504 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.535093 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7e755-1c10-4bdb-a263-563294685da3 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.547827 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.548501 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.548775 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.549042 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.549892 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.552550 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.637455 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1440/2891] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:44 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:44.641745 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.656172 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820482d-0cba-48e3-aebe-d3d2c2e57bf7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.675185 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:44.763049 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.763585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.763866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.764164 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.764768 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.770273 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.806933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1452/2892] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:44 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:44.808479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.822132 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931710-b2fd-4320-8654-554cb157e2f5 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.827509 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.828432 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.828919 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.829417 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.830277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.832197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.854423 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1441/2893] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:44.859535 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.873369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd571b-ca35-4d35-9f98-4f9dad36ed0f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.898708 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.899410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.899697 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.899977 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] PATH_INFO: `/v3/roles` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:44.900643 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:44.906951 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08817bd6-2127-4c24-a088-a5de9062029b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:44.933094 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1453/2894] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:44.934472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:44.958931 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206aed-c700-4807-8999-6a9f453ecb98 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:44.976121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:44.981142 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] REQUEST_METHOD: `PUT` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:44.981672 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:44.982123 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] PATH_INFO: `/v3/projects/f40913ed988e4d3f8164fad43a6e8508/users/1b4df8619fbc4d3bb4097fde33a400f7/roles/87b7a008126642369351bd284f3e8354` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:45.027381 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40913ed988e4d3f8164fad43a6e8508, user_id=1b4df8619fbc4d3bb4097fde33a400f7, role_id=87b7a008126642369351bd284f3e8354)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:45.030173 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:45.062269 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1442/2895] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:44 2022] PUT /identity/v3/projects/f40913ed988e4d3f8164fad43a6e8508/users/1b4df8619fbc4d3bb4097fde33a400f7/roles/87b7a008126642369351bd284f3e8354 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:45.067445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a687b8-c70c-4ce8-a902-62ea925ebb30 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:45.067723 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a687b8-c70c-4ce8-a902-62ea925ebb30 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:45.068115 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a687b8-c70c-4ce8-a902-62ea925ebb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:45.109926 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-39a687b8-c70c-4ce8-a902-62ea925ebb30 None None] MFA Rules not processed for user `5009c224905e4768b33c681c0f95b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:45.112007 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-39a687b8-c70c-4ce8-a902-62ea925ebb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:45.265103 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1454/2896] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:45.274288 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:45.276461 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:45.400074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a0eb8-1305-48ae-aec0-d670c912702a admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:45.411065 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:45.414623 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:45.414938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:45.415216 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:45.437265 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:45.442577 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:45.467329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1443/2897] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:47.392250 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca5b4-51d9-41e8-8aa4-5fcb4e6b2fdd None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:47.392859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca5b4-51d9-41e8-8aa4-5fcb4e6b2fdd None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:47.392859 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca5b4-51d9-41e8-8aa4-5fcb4e6b2fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:47.427482 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-47cca5b4-51d9-41e8-8aa4-5fcb4e6b2fdd None None] MFA Rules not processed for user `ee1094de309444bd9010b33ced748435`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:47.428826 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-47cca5b4-51d9-41e8-8aa4-5fcb4e6b2fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:47.527314 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1455/2898] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:47.535044 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:47.537514 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:47.559008 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:47.633323 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08af79-8990-4c2d-8f22-f80c7f3f2dc4 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:47.645135 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '716e5d4773d643edbce5191347661f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:47.645472 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:47.645749 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:47.646015 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:47.653709 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:47.762189 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:47.763764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:47.781314 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1444/2899] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:57:47.945278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e6866-2536-4449-bf68-159002cdcd47 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:47.945278 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e6866-2536-4449-bf68-159002cdcd47 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:47.945630 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e6866-2536-4449-bf68-159002cdcd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:47.982636 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-be4e6866-2536-4449-bf68-159002cdcd47 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:47.984508 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-be4e6866-2536-4449-bf68-159002cdcd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:47.992862 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:48.080980 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1456/2900] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:57:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:48.137761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:48.139487 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:48.236677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76540c49-fd6b-4e5d-8f86-41cd433c083d service glance] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:48.243101 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:48.243763 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:48.244026 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:48.244378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:48.260049 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:48.262124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:48.276541 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1445/2901] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:57:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:57:51.601089 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:51.601657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:51.601657 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:51.654607 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] MFA Rules not processed for user `1b4df8619fbc4d3bb4097fde33a400f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:51.655983 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:51.761543 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1457/2902] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:51.766366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:51.782783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a71f6-0540-4135-98cc-148a5f193e50 service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:51.790939 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:51.791880 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:51.792351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:51.792806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:51.793653 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:51.795978 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:51.834266 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1446/2903] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Sep 3 03:57:51 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 03:57:51.838369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:51.852169 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ba8a7-3253-4615-9d78-ff7dff5c87c3 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:51.957040 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:51.957875 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:51.958227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:51.958570 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] PATH_INFO: `/v3/users` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:51.959321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:51.961337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:52.001858 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1458/2904] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Sep 3 03:57:51 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 03:57:52.005667 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:52.020570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7474f-b1ac-4258-a3fb-a2d5f8a4bff8 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:52.027578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:52.028639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:52.029102 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:52.029561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:52.030480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:52.032470 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:52.057677 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1447/2905] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:52.062149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:52.079012 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588939a-6be9-4bc4-bff0-1b5683caf27f admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:52.107781 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:52.108481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:52.108758 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:52.109030 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] PATH_INFO: `/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89/users/db592d0cfda44b5a834310f8e9ff9226/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:52.142371 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b0267c6f674054abf0fb4b3e9ace89, user_id=db592d0cfda44b5a834310f8e9ff9226, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:52.148184 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:52.177642 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1459/2906] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:52 2022] PUT /identity/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89/users/db592d0cfda44b5a834310f8e9ff9226/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:52.178969 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:52.196158 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1a7f-13a5-4bc1-8782-72b56206a8bc admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:52.228219 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:52.302124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:52.303051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:52.303490 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:52.303937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] PATH_INFO: `/v3/roles` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:52.304756 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:52.307017 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d28cf-d0a3-4dce-9580-32732707ac8f admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:52.324313 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1448/2907] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Sep 3 03:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 03:57:52.325488 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:52.333445 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:57:52.337426 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4da2-3504-4e30-956b-54e29de01d5b admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:52.343945 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:52.345073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:52.345458 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:52.346719 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] PATH_INFO: `/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89/users/db592d0cfda44b5a834310f8e9ff9226/roles/c0d6e38a7c014c9cbdbb8dc87533008a` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:52.348385 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b0267c6f674054abf0fb4b3e9ace89, user_id=db592d0cfda44b5a834310f8e9ff9226, role_id=c0d6e38a7c014c9cbdbb8dc87533008a)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:52.351192 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:52.367524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1460/2908] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Sep 3 03:57:52 2022] PUT /identity/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89/users/db592d0cfda44b5a834310f8e9ff9226/roles/c0d6e38a7c014c9cbdbb8dc87533008a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 03:57:57.885351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167528b-5512-4f54-bc69-03197c7dc848 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:57.885951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167528b-5512-4f54-bc69-03197c7dc848 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:57.885951 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167528b-5512-4f54-bc69-03197c7dc848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:57.924383 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-1167528b-5512-4f54-bc69-03197c7dc848 None None] MFA Rules not processed for user `db592d0cfda44b5a834310f8e9ff9226`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:57:57.925645 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1167528b-5512-4f54-bc69-03197c7dc848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:58.031912 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1449/2909] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:57:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:57:58.066065 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:57:58.068737 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:57:58.203349 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6c0ff-d311-4f39-bd0d-6cb282788b0d admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:57:58.214608 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44099346-94ee-499a-8351-806e725f666f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:57:58.215424 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099346-94ee-499a-8351-806e725f666f service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:57:58.215709 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099346-94ee-499a-8351-806e725f666f service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:57:58.216244 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099346-94ee-499a-8351-806e725f666f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:57:58.296788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44099346-94ee-499a-8351-806e725f666f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:57:58.299564 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44099346-94ee-499a-8351-806e725f666f service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:57:58.313156 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1461/2910] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:57:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:58:02.070480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:02.071068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:02.071068 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:02.090976 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] MFA Rules not processed for user `0a5317994a7443fe8baeff831cd9ee81`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:58:02.092184 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:58:02.151950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1450/2911] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:58:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:58:04.113178 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf14a9-0753-4baf-9d0d-a99cfd14fe5e None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:04.113744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf14a9-0753-4baf-9d0d-a99cfd14fe5e None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:04.113917 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf14a9-0753-4baf-9d0d-a99cfd14fe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:04.135300 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-63cf14a9-0753-4baf-9d0d-a99cfd14fe5e None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:58:04.136753 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-63cf14a9-0753-4baf-9d0d-a99cfd14fe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:58:04.211134 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1462/2912] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Sep 3 03:58:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:58:04.225761 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:58:04.298826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=100 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:04.322847 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1a4ba-23c9-4fce-bf0e-c26a731c11dc None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:58:04.328118 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:58:04.328702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:04.328980 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:04.329254 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:04.340995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:58:04.343355 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:58:04.362720 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1451/2913] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 03:58:14.204010 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:14.350784 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=9 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:15.651073 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:15.651639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:15.651639 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:15.671662 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] MFA Rules not processed for user `1b4df8619fbc4d3bb4097fde33a400f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:58:15.672887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:58:15.680681 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1463/2914] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 03:58:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:58:25.666558 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:32.230834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:58:32.286845 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc384efc-77e0-424e-80b2-f63fd48d62df service swift] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:58:32.290901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:58:32.292257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:32.292257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:32.292257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:32.302701 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:58:32.304151 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:58:32.314351 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1452/2915] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:58:32.695759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:58:32.697443 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:58:32.710606 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ecc1e-0767-4dcb-a0ad-85db359ea1ad None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:58:32.714388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:58:32.715024 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:32.715299 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:32.715567 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:32.775133 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:58:32.776691 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:58:32.786498 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1464/2916] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:58:42.309995 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=26 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:42.782413 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=30 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:58:59.133141 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:58:59.142591 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7b6e7-1d36-435a-9a2d-a23e82108ce6 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:58:59.149311 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:58:59.150051 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:58:59.150328 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:58:59.150593 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:58:59.152454 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:58:59.166834 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:58:59.168277 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:58:59.179391 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1453/2917] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:59:00.501194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:59:00.511601 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78779321-75f8-4165-990b-1f401e113973 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:59:00.516149 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:59:00.516744 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:59:00.517018 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:59:00.517286 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:59:00.519421 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:59:00.533621 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:59:00.535055 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:59:00.546434 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1465/2918] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:59:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:59:09.174505 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:59:10.541406 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=6 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 03:59:49.551930 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:59:49.561872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182a7c-7597-4ea8-861a-6d4a0a93cc70 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:59:49.566431 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:59:49.567702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:59:49.567702 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:59:49.567957 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:59:49.569785 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:59:49.582707 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:59:49.584570 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaddad-1846-4706-bdcc-10e9ee509cc2 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:59:49.594106 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1454/2919] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:59:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:59:55.328088 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:59:55.341023 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5154-6dbe-468b-8539-ad48ec1f4a5f service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:59:55.347590 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:59:55.348224 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:59:55.348659 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:59:55.348916 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:59:55.350907 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:59:55.364358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:59:55.365931 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:59:55.376890 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1466/2920] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 03:59:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 03:59:56.790037 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:59:56.790561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:59:56.790561 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:59:56.842401 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] MFA Rules not processed for user `156bd0c3f4b8489888de3ca7b7342e09`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 03:59:56.843578 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 03:59:56.861790 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1455/2921] 10.222.0.25 () {58 vars in 967 bytes} [Sat Sep 3 03:59:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 03:59:56.907304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 03:59:56.928855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd81062-db3e-4c5e-9ba9-460caa7a832d service nova] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 03:59:56.934191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '323c45a05bd241458486b8919c93dd2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 03:59:56.934788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 03:59:56.935096 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 03:59:56.935369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 03:59:56.943896 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 03:59:56.945465 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 03:59:56.953947 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1467/2922] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Sep 3 03:59:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 04:00:03.803489 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:03.805217 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:03.875955 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eadbb-0797-4987-81f5-4b7f99edc0a0 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:03.880762 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:03.881346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:03.881668 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:03.882004 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] PATH_INFO: `/v3/users/0a5317994a7443fe8baeff831cd9ee81` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:03.883196 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5317994a7443fe8baeff831cd9ee81)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:03.885137 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:03.940901 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5317994a7443fe8baeff831cd9ee81'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:03.941564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5317994a7443fe8baeff831cd9ee81'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:03.954346 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5317994a7443fe8baeff831cd9ee81'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:03.961658 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5317994a7443fe8baeff831cd9ee81'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:03.962306 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5317994a7443fe8baeff831cd9ee81'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:03.976867 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1456/2923] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 04:00:03 2022] DELETE /identity/v3/users/0a5317994a7443fe8baeff831cd9ee81 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:00:04.159303 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:04.170481 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad446-e062-4b36-8a7c-73d327c98273 service swift] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:04.227191 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:04.227814 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:04.228092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:04.228357 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] PATH_INFO: `/v3/projects/766e95a843fd4c658750c125427135cf` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:04.229820 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=766e95a843fd4c658750c125427135cf)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:04.231789 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:04.279128 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766e95a843fd4c658750c125427135cf', 'request_id': 'req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.288138 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766e95a843fd4c658750c125427135cf', 'request_id': 'req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.294227 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766e95a843fd4c658750c125427135cf', 'request_id': 'req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.305337 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] The token cache is being invalidate because project 766e95a843fd4c658750c125427135cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 04:00:04.305623 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.306316 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.307954 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1468/2924] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 04:00:04 2022] DELETE /identity/v3/projects/766e95a843fd4c658750c125427135cf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:00:04.312271 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:04.313759 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:04.327378 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2485eaf-5bfe-4869-8fd4-d44ad711bfb2 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:04.352263 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:04.389548 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:04.390176 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:04.390449 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:04.390714 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] PATH_INFO: `/v3/users/ee1094de309444bd9010b33ced748435` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:04.391941 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1094de309444bd9010b33ced748435)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:04.393859 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:04.442465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1094de309444bd9010b33ced748435'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.443121 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1094de309444bd9010b33ced748435'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.455091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1094de309444bd9010b33ced748435'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.462599 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1094de309444bd9010b33ced748435'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.463344 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-1b990075-9fc5-4594-a2d2-37af96fba74c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1094de309444bd9010b33ced748435'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.478283 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1457/2925] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 04:00:04 2022] DELETE /identity/v3/users/ee1094de309444bd9010b33ced748435 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:00:04.656757 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:04.658163 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:04.673902 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af88d8b-6094-4bc5-9cf2-fe9ae0e6ed21 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:04.727677 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:04.736997 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:04.737796 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:04.738232 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:04.738518 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] PATH_INFO: `/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:04.739765 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a66ef2ac1840d2b030b0f059bebf74)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:04.741760 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:04.789017 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a66ef2ac1840d2b030b0f059bebf74', 'request_id': 'req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.796973 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a66ef2ac1840d2b030b0f059bebf74', 'request_id': 'req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.802466 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a66ef2ac1840d2b030b0f059bebf74', 'request_id': 'req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.813180 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] The token cache is being invalidate because project a5a66ef2ac1840d2b030b0f059bebf74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93068) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 04:00:04.813460 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.814086 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:00:04.815585 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1469/2926] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 04:00:04 2022] DELETE /identity/v3/projects/a5a66ef2ac1840d2b030b0f059bebf74 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:00:14.474317 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:14.810369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:35.291920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] REQUEST_METHOD: `POST` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:35.292465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:35.292465 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:35.312821 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.auth.core [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] MFA Rules not processed for user `1b4df8619fbc4d3bb4097fde33a400f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93069) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 04:00:35.314091 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:35.375095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1458/2927] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 04:00:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 04:00:35.715759 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:35.718145 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:35.783129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1d1eb-add1-42ae-8db6-5cc3c2046d6c admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:35.787755 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:35.788388 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:35.788663 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:35.788933 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:35.790857 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:35.800524 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:35.801996 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:35.810201 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1470/2928] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:00:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:00:37.299921 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:37.301564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:37.375806 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee866e1-abb5-459f-8611-e7ed9d6701d6 None None] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:37.381087 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:37.381705 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:37.381972 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:37.382275 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:37.384074 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:37.401369 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:37.403468 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:37.416366 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1459/2929] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:00:45.806836 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=26 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:47.410950 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=57 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:00:52.523170 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:00:52.525132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:52.607335 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990e2d0-d1f3-4dfa-8b62-0d86b2338f80 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:00:52.612879 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:00:52.613513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:00:52.613800 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:00:52.614067 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:00:52.615874 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:00:52.628505 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:00:52.630100 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60b3bb-8cda-42fa-985e-366ab618b1ad service nova] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:00:52.640513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1471/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:00:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:01:02.636433 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=34 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:02:21.619974 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:02:21.622467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:02:21.637730 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0c8d-7af2-469f-9fc9-8f5823e35bb2 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:02:21.644273 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:02:21.644938 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:02:21.645212 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:02:21.645480 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:02:21.647506 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:02:21.659596 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:02:21.661178 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:02:21.671177 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1460/2931] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:02:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:02:31.666764 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:02:34.077602 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec164-812c-40af-8708-2d1306524208 None None] REQUEST_METHOD: `POST` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:02:34.078155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec164-812c-40af-8708-2d1306524208 None None] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:02:34.078155 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec164-812c-40af-8708-2d1306524208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:02:34.105483 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.auth.core [None req-2dfec164-812c-40af-8708-2d1306524208 None None] MFA Rules not processed for user `1b4df8619fbc4d3bb4097fde33a400f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93068) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 04:02:34.107378 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2dfec164-812c-40af-8708-2d1306524208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:02:34.118254 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1472/2932] 10.222.0.25 () {58 vars in 958 bytes} [Sat Sep 3 04:02:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 04:02:44.096381 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=3 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:02:51.220257 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:02:51.234021 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e91e-58fd-43d7-9810-c9edcd242f63 service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:02:51.240124 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e0e6be0fc61491fa25193834b4fa8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:02:51.240736 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:02:51.241012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:02:51.241279 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:02:51.243168 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:02:51.256937 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:02:51.258838 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:02:51.270331 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1461/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:02:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 04:03:01.265598 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:18.336816 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfec164-812c-40af-8708-2d1306524208 None None] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:18.338369 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-2dfec164-812c-40af-8708-2d1306524208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:18.352330 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfec164-812c-40af-8708-2d1306524208 None None] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:18.356946 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:18.357573 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:18.357866 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:18.358129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:18.360146 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:18.371864 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:18.373304 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:18.382517 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1473/2934] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:19.459793 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:19.470291 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085c0b-4ac0-4da0-9128-bc5fe707a300 service nova] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:19.473873 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:19.474484 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:19.474758 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:19.475059 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:19.477683 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:19.491872 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:19.493424 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:19.505080 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1462/2935] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:20.694321 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:20.703788 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6b140-8b60-4922-a675-94566bb11bc0 service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:20.708092 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:20.708688 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:20.708974 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:20.709243 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:20.711547 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:20.725358 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:20.727332 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:20.739042 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1474/2936] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:29.500061 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=6 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:30.734439 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=12 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:32.022479 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:32.031825 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddff-5246-4879-ab35-ec465ddea4a1 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:32.036035 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c0ba8119d54212a8c142353d27cd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:32.036554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:32.036826 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:32.037090 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:32.038960 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:32.102716 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:32.104188 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:32.113783 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1463/2937] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:35.233513 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:35.247807 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6b6eb-9c78-4aa4-841e-a4021a0be26c service neutron] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:35.262939 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:35.263712 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:35.263824 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:35.264098 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:35.265911 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:35.279867 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:35.281519 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:35.293014 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1475/2938] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:38.199581 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:38.212086 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136886d-3b90-4114-ba16-80b325960dd4 service neutron] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:38.218282 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:38.218920 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] REQUEST_METHOD: `GET` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:38.219197 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:38.219467 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:38.221498 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:38.234399 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:38.237357 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:38.249587 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1464/2939] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:04:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:04:38.292079 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:38.294194 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:38.309310 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64aeb99-7d1f-4e00-8ddc-fd3e810812f9 service placement] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:38.313887 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:38.314534 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:38.314829 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:38.315132 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] PATH_INFO: `/v3/users/1b4df8619fbc4d3bb4097fde33a400f7` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:38.316359 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4df8619fbc4d3bb4097fde33a400f7)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:38.318341 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:38.376083 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4df8619fbc4d3bb4097fde33a400f7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.387855 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4df8619fbc4d3bb4097fde33a400f7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.394550 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4df8619fbc4d3bb4097fde33a400f7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.395339 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4df8619fbc4d3bb4097fde33a400f7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.395987 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4df8619fbc4d3bb4097fde33a400f7'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.410351 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1476/2940] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 04:04:38 2022] DELETE /identity/v3/users/1b4df8619fbc4d3bb4097fde33a400f7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:04:38.597564 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:38.614292 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4fb7-deff-4b11-90a8-7722f889f21e service placement] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:38.680195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:38.680752 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:38.681031 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:38.681303 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] PATH_INFO: `/v3/projects/f40913ed988e4d3f8164fad43a6e8508` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:38.682503 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40913ed988e4d3f8164fad43a6e8508)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:38.684642 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:38.739110 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40913ed988e4d3f8164fad43a6e8508', 'request_id': 'req-bc79e0e9-9349-4274-a57c-d32238cba965'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.748486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40913ed988e4d3f8164fad43a6e8508', 'request_id': 'req-bc79e0e9-9349-4274-a57c-d32238cba965'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.754526 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40913ed988e4d3f8164fad43a6e8508', 'request_id': 'req-bc79e0e9-9349-4274-a57c-d32238cba965'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.766486 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] The token cache is being invalidate because project f40913ed988e4d3f8164fad43a6e8508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 04:04:38.766778 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.767554 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.769511 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1465/2941] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 04:04:38 2022] DELETE /identity/v3/projects/f40913ed988e4d3f8164fad43a6e8508 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:04:38.773289 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:38.774928 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:38.788626 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3c2a-c598-454d-b5b6-9f859f8bbd6a admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:38.850461 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:38.851120 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:38.851711 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:38.851981 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] PATH_INFO: `/v3/users/db592d0cfda44b5a834310f8e9ff9226` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:38.853121 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db592d0cfda44b5a834310f8e9ff9226)` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:38.855053 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:38.903311 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db592d0cfda44b5a834310f8e9ff9226'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.914082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:38.916392 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db592d0cfda44b5a834310f8e9ff9226'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.924491 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db592d0cfda44b5a834310f8e9ff9226'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.925308 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db592d0cfda44b5a834310f8e9ff9226'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.925913 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.notifications [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db592d0cfda44b5a834310f8e9ff9226'} {{(pid=93068) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:38.939568 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1477/2942] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Sep 3 04:04:38 2022] DELETE /identity/v3/users/db592d0cfda44b5a834310f8e9ff9226 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:04:39.118540 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Authenticating user token {{(pid=93069) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:04:39.120329 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.fernet_utils [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93069) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:04:39.137789 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79e0e9-9349-4274-a57c-d32238cba965 admin admin] Validating token access rules against request {{(pid=93069) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:04:39.145742 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:39.200095 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5009c224905e4768b33c681c0f95b0f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1fa3555aedbc4a4eaa2e41bd6d2eeab3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93069) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:04:39.200824 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:04:39.201012 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] SCRIPT_NAME: `/identity` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:04:39.201286 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] PATH_INFO: `/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89` {{(pid=93069) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:04:39.202637 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b0267c6f674054abf0fb4b3e9ace89)` {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:04:39.204644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] RBAC: Authorization granted {{(pid=93069) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:04:39.261644 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b0267c6f674054abf0fb4b3e9ace89', 'request_id': 'req-9e277419-9107-4aec-8272-98ee88d02ade'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:39.270860 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b0267c6f674054abf0fb4b3e9ace89', 'request_id': 'req-9e277419-9107-4aec-8272-98ee88d02ade'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:39.276542 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b0267c6f674054abf0fb4b3e9ace89', 'request_id': 'req-9e277419-9107-4aec-8272-98ee88d02ade'} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:39.287888 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] The token cache is being invalidate because project c0b0267c6f674054abf0fb4b3e9ace89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93069) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 04:04:39.288195 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:39.288952 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG keystone.notifications [None req-9e277419-9107-4aec-8272-98ee88d02ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93069) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 04:04:39.290538 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: [pid: 93069|app: 0|req: 1466/2943] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Sep 3 04:04:39 2022] DELETE /identity/v3/projects/c0b0267c6f674054abf0fb4b3e9ace89 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 04:04:48.936129 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:04:49.284639 dv-cinder-799663-12-iscsi devstack@keystone.service[93069]: DEBUG dbcounter [-] [93069] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93069) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 04:07:30.595952 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Authenticating user token {{(pid=93068) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 04:07:30.597766 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:07:30.741255 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7183b95-edac-4af5-bbc9-85678a146be7 admin admin] Validating token access rules against request {{(pid=93068) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 04:07:30.805239 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15badc5a92d745b0896aed0f7c582e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa493a790af417a97ccfb7b24b5f401', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93068) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 04:07:30.806143 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] REQUEST_METHOD: `GET` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 04:07:30.806410 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] SCRIPT_NAME: `/identity` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 04:07:30.806762 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93068) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 04:07:30.810082 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.fernet_utils [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93068) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 04:07:30.826147 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 04:07:30.827769 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ad7f4-b773-4783-ba72-e19cacc23929 service placement] RBAC: Authorization granted {{(pid=93068) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 04:07:30.838056 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: [pid: 93068|app: 0|req: 1478/2944] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Sep 3 04:07:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 04:07:40.833785 dv-cinder-799663-12-iscsi devstack@keystone.service[93068]: DEBUG dbcounter [-] [93068] Writing DB stats keystone:SELECT=89 {{(pid=93068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}